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EXECUTIVE SUMMARY 



Many federal education programs support technical assistance to state departments of 
education, school districts, and schools. This report discusses the accomplishments and limitations of 
a recent initiative in federally supprrted assistance to schools. The lessons of this initiative can be 
applied in two laws that are soon to be reauthorized, the Elementary and Stvxmdary Education Act 
(ESEA) and the authorizing legislation for the Office of Educational Research and Improvement of the 
U.S. Department of Education (ED;; both laws address the organization and operations of technical 
assistance. 

The Nine-Site Program Improvement Initiative evaluated in this report was a three-year 
venture in which federal contractors provided technical assistance to schools. The contractors- 
Chapter 1 Technical Assistance Centers (TACs) and Rural Technical Assistance Centers (RTACs)- 
received about $60,000 per year for their work with each site, which included several schools. The 
participating schools had programs funded by Chapter 1 of ESEA that had been identified as needing 
improvement on the basis of inadequate gains in student performance. Five sites were large urban 
districts: Baltimore, Chicago, Detroit, Los Angeles, and Dade County, Florida. Three sites included 
relatively small, isolated rural schools: Pike County, Kentucky; a cluster of schools in southeastern 
Iowa; and nine schools in six districts in the Mississippi Delta. Seven schools administered by the 
Bureau of Indian Affairs (BIA) in the U.S. Department of Interior comprised the ninth site. 

The Nine-Site Initiative offers lessons about ways of organizing and conducting technical 
assistance that goes directly to indh toual schools and that is sustained over a reasonably long period 
of time. On the positive side, this e /Uuation identified several accomplishments of the initiative: 

• At the end of the thi- ! year of the initiative in all of the sites, there was evidence of 
increased understanding of Chapter 1 regulations and requirements, improvements in 
the quality and undei binding of achievement test data, and increased understanding 
and experimentation wth new instructional techniques. 

• In several sites, participation in technical assistance activities appeared to contribute to 
increased collegiality and professional confidence among teachers. 

• In two sites, technical assistance contributed to changes in district administration of 
Chapter 1. 
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• Overall, teachers, principals, and district and state administrators who were familiar 
with the assistance activities gave them high marks and felt that the TAC/RTACs 
responded to local problems and concerns. They also appreciated the materials the 
TAC/RTACs provided, particularly information about programs and practices 
operating in other places around the country. 

Nevertheless, despite these positive outcomes and the perceptions of participants, a central 
finding of this evaluation is that the technical assistance had limited impact in these schools. Reasons 
for the lack of significant impact include the following: 

• Technical assistance was linked to an unpopular and incompletely understood federal 
mandate, with the result that it was difficult to initiate any activities and there was 
little interest in and commitment to them once they began. 

• Technical assistance content and strategies were, for the most part, determined by the 
TAC/RTACs and local administrators with little or no input from teachers. The 
activities were primarily intended to convey information, but not necessarily to 
contribute to professional development or an informed critique of current programs 
and practices in individual schools. 

• Technical assistance was not strongly linked to other reform initiatives, nor was it 
guided by local visions of how schools that serve disadvantaged students might be 
better. 

• There was little support and encouragement for teachers to participate in the assistance 
activities or to work on related improvement tasks at other times. 

The Nine-Site Initiative began under conditions that federal officials expected to be conducive 
to success. Several of the sites were already engaged in ambitious reforms (e.g., the changes in 
Chicago school governance, site-based management in Dade County, and the Kentucky Educational 
Reform Act). The mandate for Chapter 1 program improvement had been imposed on all the 
participating schools, bringing with it a requirement for school-level improvement plans. The 
TAC/RTACs were veteran providers of assistance on assessment and program improvement. 

On the other hand, some circumstances surrounding the initiative were less auspicious. In an 
evaluation that tracked the early course of the initiative, we found enormous confusion about the 
duration of ED* s support for the initiative and about the expectations for what was to be 
accomplished. The earlier evaluation also found that schools in many of the sites were coping with 
conditions of extreme disadvantage in their communities, which made it difficult to concentrate on 
change and improvement despite the overwhelming need. Ironically, the presence of other refonn 
initiatives in some sites, instead of creating momentum that could help carry Chapter 1 program 



improvement, instead impeded start-up of the nine-site activities because state and local reform 
initiatives took precedence over Chapter 1 program improvement. All these circumstances combined 
to dampen the potential local commitment to the technical assistance activities. 

This absence of commitment, combined with the fact that, in eight of the nine sites, 
participation was mandatory, meant that the TAC/RTACV had to spend considerable time selling the 
initiative and themselves to the schools. Furthermore, two sites, Los Angeles and Mississippi, were 
served by a TAC or RTAC that lost its contract after the first year. In each case, the new contractor 
had to repeat the work of building rapport. 



The Context and the Services 

Through on-site interviews with teachers, principals, district administrators, and technical 
assistance providers, along with a review of detailed monthly reports from the TAC/RTACs, this 
evaluation examined the local contexts in the sites and the technical assistance services provided 
through the Nine-Site Initiative. 



The Siltti Not Pr*dlnwn#d to Ua* Anlttance 

The nine sites participating in this initiative ran the gamut from immense urban districts to 
isolated rural settings; most of the schools had high levels of poverty among their students. Despite 
the mandate to improve their Chapter 1 programs, few schools embarked on this initiative wfth a 
drive for reform in Chapter 1-not the least because for many of them the clash of multiple reform 
initiatives was a continuing worry. 

• The requirement for Chapter 1 program improvement did not, by itself, create fertile 
ground for assistance in any of the schools. Typically, teachers and principals were 
mildly resentful that their school had been identified for improvement and wanted the 
designation to go away. 

• In the urban sites, large, complex bureaucracies, labor disputes, and impending 
budget shortfalls made it difficult for the TACs to get the attention of local 
administrators and school staffs. 

• In the rural sites, by contrast, the relative absence of competing programs and 
pressing problems meant that the technical assistance activities received greater 
support. 
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Participation: Varying but Modest Intensity 



Although the Nine-Site Initiative was considered a departure from previous federal assistance 
because it would provide intensive services to individual schools, local participants typically did not 
experience the services as intensive. Initially, many schools were included, but in the third year, 
even with fewer schools engaged in the initiative, a typical school received only a few person-days of 
on-site help. Furthermore, these on-site days represent the assistance provider's time, not the 
individual recipients' time; under the varying local arrangements only a small proportion of 
participating teachers actually experienced more than a few hours of assistance or saw the assistance 
provider as often as monthly. (The appendices to the report contain detailed information about the 
content and amount of assistance provided.) 

• During the first year of the Nine-Site Initiative, the total number of participating 
schools was 68, with the number of schools receiving services ranging from five to 
ten per site. The total number of participating schools declined by one-third by the 
beginning of the third year of the initiative. 

• In the third year of the initiative, the mean amount of TAC/RTAC assistance to 
individual participating schools was 40 hours, with a median of 43 hours. The range 
was from 81 hours at a school in Baltimore down to 12 hours at a school in Chicago. 
In four sites, TAC/RTAC s achieved some economies of scale by providing assistance 
to several schools in a central location. 

• In three sites, TAC/RTACs provided assistance to each individual school every month 
of the year. In the other sites, individual schools did not receive on-site assistance 
every month; some schools received as few as two or three visits from TAC/RTAC 
staff during the year. 

• In Baltimore, Iowa, and Los Angeles, technical assistance was targeted to clearly 
defined, small groups of school staff, who received relatively high levels of 
assistance. At other sites, teacher participation was more broadly distributed; as a 
result, more teachers participated in assistance activities, but individual teachers 
received very little assistance. 



Content; Information Trgmmfodnn 

By and large, the nine-site assistance disseminated new information to participants. To a 
lesser extent, it trained them in applying new skills. It did little to help them tie the new information 
or skills into a broader vision of instructional improvement. Although the content of the technical 
assistance varied across sites and individual schools, the common denominators included a focus on 
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instruction, Chapter 1 requirements, and assessment. In several sites, the content shifted and evolved 
over time; a clear, consistent focus was the exception rather than the rule. 

• In all sites, the content of technical assistance usually included: (1) training in new 
instructional practices, especially "hot topics' 1 like cooperative learning, whole 
language, and math manipulative*; (2) assistance with Chapter 1 program 
administration and the interpretation and use of student test data; and (3) assistance or 
training related to local or state initiatives, 

• The most ambitious TAC/RTAC efforts to introduce innovations in classroom 
instruction attempted to ground teachers in the research and rationales underlying 
whole language, literature-based language arts, or the teaching of problem solving and 
other advanced skills in mathematics. Frequently, assistance activities had the more 
limited goal of introducing information about discrete instructional practices that 
teachers could use immediately. 

• In most sites, TAC/RTAC assistance attempted to strengthen school-level efforts to 
meet the requirements of state and local reforms; in several sites, TAC/RTAC 
assistance included a modest amount of help in developing and evaluating schoolwide 
projects. 



Varying Stajgte ftr Organism md Delivering Trehntol Assistance 

In the absence of specific direction about how to organize their services, the TAC/RTACs 
developed different ways of working with the sites. The approaches and their results varied greatly. 

• Where TAC/RTAC activities involved groups of schools, they achieved some 
economies of scale, providing more contact hours to schools and teachers than visits 
to individual schools alone would allow. In other sites, TAC/RTACs pursued a 
strategy of intensive attention to a small number of teachers, working on classroom 
instruction through demonstration teaching and coaching. 

• Technical assistance in Iowa, which involved teams of teachers and administrators 
from each school participating in a series of activities that were connected to a clear 
framework, reflected the most thoroughly developed strategy for organizing and 
providing technical assistance. 

• In sites where one-on-one sessions were the dominant TAC/RTAC strategy, sessions 
tended to be discrete, isolated events with little relation to one another. 

• In sites where a clear content focus and a well-defined strategy for delivering services 
guided the technical assistance process, the assistance was much more likely to be 
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cumulative over the three years of the initiative. In sites without a clear focus, 
technical assistance tended to be tentative and reactive. 

We found a few examples of TAC/RTAC cooperation with other external assistance 
providers, but no extensive collaboration. 



Conclusions and New Directions for Technical Assistance 

The Nine-Site Initiative had its successes, especially as perceived by participants at the sites. 
The TACs and RTACs understood local demands for assistance and worked hard to meet them. One 
indicator of their success is that in five of the nine sites, negotiations are under way or contracts have 
been signed to extend TAC/RTAC assistance into 1993-94. Our findings about the amount, content, 
organization, and outcomes of the assistance lead to several favorable conclusions: 

• The technical assistance activities and services generally met teachers 1 and principals 1 
standards for quality. 

• The TAC/RTACs provided the schools with a large amount of information and 
materials about instructional practices, curriculum, and student assessment. 

• Many participating teachers reported that they had experimented with the new 
stra'egies in their classrooms and found that these strategies "worked M --they held 
students' attention, they relieved teachers' boredom, and they offered new ideas for 
lesson plans. 

• Follow-up visits, although usually limited in duration and frequency, gave > achers a 
chance to review progress and seek additional advice on how to use the information. 
Repeated visits over the three-year period, even though they were not always frequent 
or regular, helped set TAC/RTAC assistance apart from assistance that teachers and 
principals reject as not helpful. 

• In several sites and schools, TAC/RTAC assistance extended the standard model in 
two additional ways. First, assistance was targeted to stable groups of teachers whose 
participation was sustained for a long period of time. Second, the assistance topics 
were cumulative-each individual activity was part of a framework that connected it to 
a broader vision. 

Despite some real contributions, though, the Nine-Site Initiative had a limited impact on 
participating schools specifically because the assistance conformed to conventional models of external 
assistance and school change. The bulk of the technical assistance was organized around 
presentations, workshops, and other dissemination activities given by experts to impart skills to 
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teachers. It was designed to support training and the transfer of information and skills, not problem 
solving. If the intent of the initiative was to generate serious attempts at Chapter 1 (or broader) 
improvement, then this traditional model of technical assistance wa; not a good choice. 

No one should have expected that TACs and RTACs would select an alternative mode of 
assistance. One reason is that prior experience with technical assistance leads teachers and principals 
to expect a certain type of service, moit likely one that revolves around a set menu of options for 
workshops and presentations related to specific instructional techniques and ways to install them into 
daily lesson plans (Fullan, 1991). In addition, given the structure erf ihe Nine-Site Initiative— 
particularly its link to Chapter I program improvement requirements-its fairly hasty start-up, and the 
confusion regarding ED's goals in the initiative, the TAC/RTACs had little reason to rethink their 
traditional approach to technical assistance. Instead, they chose the rational and reasonable course of 
following the technical assistance model of the day. Their client (ED) implicitly demanded it—by not 
directing the TAC/RTACs to try different or novel approachcs-«nd their target audience (schools) 
expected and happily received it. 

The organization, content, and ultimately the impact of technical assistance were also limited 
by the fact that few participants, if any, had clear, long-range visions or plans for changing their 
schools. Although the Chapter 1 requirement for program improvement plans was intended to 
stimulate such visions, in fact the school plans did not reflect careful thought about curriculum, 
instruction, assessment, or staff development, nor did the plans appear to have emerged from 
extensive participation and deliberation by teachers and other members of the school staffs. 
Administrators and teachers in all of the schools had little understanding of Chapter 1 program 
improvement at the time the Nine-Site Initiative began. For most of them, it represented a 
bureaucratic requirement that did little to inspire serious thinking about opportunities and options for 
change. As we learned, the goal was to H to get out of program improvement.* 4 Some schools did test 
out during the second or third year of the initiative. A few schools that demonstrated gains did not 
sustain them and were identified a second time. Other schools remained in program improvement 
status at the end of the third year. We did not find any clear evidence to suggest that the changes in 
program improvement status were linked to anything that the schools or the TAC/RTACs did in the 
way of program improvement. 

In the absence of ambitious visions or incentives to develop them, the TAC/RTACs provided 
information and advice about alternative instructional techniques and some opportunities for feedback 
after preliminary classroom experiments. Teachers searching for new techniques welcomed and 
valued assistance that provided such techniques; however, this assistance did not push them towards 
new directions in practice or new views of their professional roles. Therefore, it is not surprising that 
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the assistance did not lead to many noticeable changei in schools' capacities to change or to be self- 
reflective. 

If ED wants to use school-level technical assistance to contribute to a more fundamental 
redesign of education for disadvantaged studenti, the experience of this initiative suggests several 
possible new directions for future technical assistance efforts. 

1. Through competitive procurements and reporting requirements, rive assistance providers 
incentives to go beyond conveying information about successful practice and to foster 
profregjongl devdopmfflt gPagfagS- 

Federal support for all the providers of technical assistance should reflect clear expectations 
about the purposes of the assistance. By the same token, because assistance that nurtures self- 
reflection and critical thinking is largely uncharted territory-few providers and probably fewer federal 
officials know how to do it— the government should avoid excessive prescription about the content and 
methods of assistance. ED could encourage or require potential contractors to propose alternative 
models of technical assistance and to explain why the models are appropriate for intended target 
audiences. Potential contractors also could be encouraged or required to provide evidence that 
potential target audiences are disposed to use the alternative forms of assistance. Indeed, an important 
task in providing new kinds of technical assistance will be to help recipients develop the capacity to 
use it. A revised reporting system also would require contractors to provide the quantitative 
information necessary to support observations and assertions about implementation and effectiveness. 

2. Create a technical assistance network that connects to and leverages state and local supp ort 
for professional development . 

Technical assistance in support of school problem solving must go hand in hand with more 
ambitious and unconventional forms of professional development. In fact, even conventional technical 
assistance demonstrates the need for connection with local professional development. Even when 
assistance is available to participants at no cost for the services, there are costs such as paying 
substitutes to cover classes, paying stipends for participation during non-school time, or simply 
allocating normal staff development time to the assistance activities. The size of these commitments 
grows rapidly as they include opportunities for teachers to continue working together between visits 
by external as sisters. Requiring local contributions to support participation in technical assistance and 
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related improvement activities will increase the likelihood that these efforts will be sustained over 
time. 



3. Create jgaaanSS TrtWflrfr ifff "riiraifv around the mes or tonics, rather than 
categorical BEaBBBMb ftlH 1 ittdllding lv >th process a nd content specialists 

The experience of the Nine-Site Initiative demonstrated the problems that external assisters 
face when their assistance is linked to a program that is not central to local improvement efforts. A 
more compelling foundation for an assistance network would be the aim of fostering ambitious reform 
in the name of a broader goal-such as bringing all students to high levels of academic performance. 

By offering assistance from specialists in both process and content, policymakers would 
recognize that both are needed. Process specialists could help schools build norms of collegiality and 
cooperation and define goals, problems, and improvement tasks. Content specialists would then be 
available, at the request of schools and districts, to help solve the problems and carry out the tasks. 

4r Integrate bsh totowtogre jute tire tedalal agateteira rateflrk- 

Technology can help in technical assistance through conveying large amounts of information 
and facilitating communication among schools and technical assistance providers. Despite the 
increasing sophistication of the new technologies and relatively easy access to them, however, this 
alternative should be pursued with caution. Most schools in the Nine-Site Initiative do not have the 
hardware and software necessary to take advantage of anything more than a routine electronic 
dissemination system. Further, even when they are available, school staff do not use them. 

S. SuDDOrt Ciptritv hnllriffiy fnr flg nrnviriArs nf tarhnlr*! agSlgjlKS- 

A central theme in the alternatives is that future technical assistance programs will provide 
services that are vastly different from those that are provided now. As we saw in the Nine-Site 
Initiative, a shift in focus to school-level assistance combined with the need to establish and maintain 
long-term relationships with schools and districts were learning experience* for the TAC/RTACs. 
The alternative assistance strategies discussed here represent even greater changes in perspective and 
practice. Therefore, we expect that there will be a need for training and staff development 
opportunities to prepare technical assistance providers to carry out their new responsibilities. The on- 
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line information systems being developed and used by the TAC/RTACs are but one example of 
internal capacity building. Networking activities among the Eisenhower National Regional Consortia 
for Mathematics and Science Education are a second example. More focused in-house staff 
development activities, including training and staff seminars, are a third possibility. 

In conclusion, although the Nine-Site Initiative affected a significant number of teachers, 
principals, district administrators, and technical assistance providers by involving them in an 
experiment in improving education for disadvantaged students, our evidence indicates that for most 
participants the impact was not very deep and will probably not last very long. The primary reason 
for the lack of significant, sustained impact W t\:-r. th? £-Siu, initiative and the-assistance it 
supported were guided by a limited view of how schools change. Thus, the challenge for future 
technical assistance programs is not to figure out ways to make the traditional approaches more 
effective. Instead, the challenge is to create and support technical assistance programs that recognize 
and address the complexity of school change and the professional needs and capabilities of educators. 
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I. INTRODUCTION 



Many federal programs and contractors provide technical assistance to state departments of 
education, school districts, and schools. For the reauthorization of the Elementary and Secondary 
Education Act (ESEA) and ED's Office of Educational Research and Improvement (OERI), 
considerable attention has focused on policy options for reorganizing ED's technical assistance 
programs to increase their efficie^^' and effectiveness, Qv& question has uten concerns the 
value of school-level technical assistance. The recently concluded Nine-Site Program Improvement 
Initiative in Chapter 1 has demonstrated some contributions and pitfalls of such assistance; this report 
analyzes its lessons. 

The Nine-Site Program Improvement Initiative was intended to explore the potential 
contribution of technical assistance in schools identified for Chapter 1 program improvement under 
the provisions of the Augustus F. Hawkins-Robert T. Stafford Elementary and Secondary School 
Improvement Amendments of 1988 (P.L. 100-297). The initiative supported assistance that was more 
intensive and of longer duration than most assistance and staff development programs available to 
schools that serve disadvantaged students. Therefore, as ED looks ahead to redesigning its technical 
assistance enterprise, it may be useful to look back at the lessons that can be learned from the Nine- 
Site Initiative. Although the initiative represents a very small portion of ED's investment in technical 
assistance-less than one-half of 1 percent in fiscal year 1992— it is one of the few long-term, school- 
level assistance activities sponsored by ED and the only one in support of a categorical program. 

At one time or another and in varying degrees, assistance provided under this three-year 
initiative served virtually every purpose addressed by federal technical assistance: 

• Helping grantees and other recipients of federal funds understand the legal aspects of 
categorical programs (e.g., informing them about regulations, assisting them in 
meeting reporting requirements) 

• Assisting state departments of education, school districts, and schools with specific 
technical tasks (e.g., student assessment, program evaluation) 

• Building the capacity of these agencies to carry out the technical tasks themselves 

• DiiMmlniting information on effective practices 

• Facilitating the development of lateral networks (e.g., among Chapter 1 schools) 
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• Working intensively with schools (or others) to help them define and solve their own 
educational problems 

This report presents the findings from the third-year evaluation of the Nine-Site Initiative and 
discusses their implications for the operation of federally sponsored technical assistance programs. 
The remainder of this introduction describes the Nine-Site Initiative, summarizes the key findings 
from a previous evaluation that we conducted at the mid-point of the initiative, and outlines oir 
approach to the ihird-yeav evaluation. The second section describes the technical assistance provided 
in the nine sites; the third section examines changes in schools over the three years of the Nine-Site 
Initiative and the impact of the assistance on teachers and principals, school organization, district 
administration of Chapter 1, and student outcomes. The last section offers some concluding 
observations about the reasons for the particular organization, content, and strategies of the assistance 
provided under the Nine-Site Initiative and discusses possible new directions for future technical 
assistance programs. 

The Nine-Site Program Improvement Initiative 

In 1990, ED established the Nine-Site Program Improvement Initiative to learn more about 
how federally sponsored technical assistance in addition to that normally provided by school districts 
and state departments of education could help the schools identified for Chapter 1 program 
improvement. The initiative was designed as a multi-year effort in which ED's Chapter 1 Technical 
Assistance Centers (TACs) and Rural Technical Assistance Centers (RTACs) would provide technical 
assistance for program improvement to schools in the targeted sites. 

ED selected nine sites, each of which offered formidable challenges to external assisters. (See 
Exhibit 1.) Five sites were large urban districts: Baltimore, Chicago, Detroit, Los Angeles, and 
Dade County, Florida. Three sites included relatively small, isolated rural schools: Pike County, 
Kentucky; a cluster of schools in southeastern Iowa; and nine schools in six districts in the Mississippi 
Delta. 1 Seven schools funded by the Bureau of Indian Affairs (BIA) in the U.S. Department of 
Interior comprised the ninth site. 2 A total of 68 schools, selected by school districts or state 



1 At the beginning of the Nine-Site Initiative, there were seven districts in the Mississippi Delta 
site; two of these districts merged during the course of the initiative. 

2 BIA schools may be operated by the BIA by tribes under contracts or grants from the BIA. Six 
of the schools in this site are operated by the bureau and one is a grant schc oL The staff of the six 
bureau-operated schools are all federal employees. 

2 
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departments of education and, in some sites, in conjunction with the TAC'RTACs, formed the initial 
group of schools that received services. The number of participating schools in each site ranged from 
a high of ten in Chicago to five in Kentucky. In Baltimore, Dade County, *nd Detroit, the TACs 
provided assistance to a Chapter 1 program serving religious-school students. Almost all of the 
participating schools were elementary schools, although there were a few middle or junior high 
schr :N and 'mo high schools. Once the sites were chosen, ED added about $60,000 per year to 
existing TAC/RTAC contracts to support the services in each site. This amount was to cover all on- 
site assistance, preparations for assistance, and TAC/RTAC travel expenses. (As the map in Exhibit 
1 illustrates, all of the TAC/RTACs are located several hundred miles from the sites included in the 
Initiative. In addition, schools in the rural sites were quite spread out-up to 80 miles apart in 
M ississippi-so travel costs and staff time spent traveling were no small matter in planning and 
scheduling technical assistance services.) The modifications to the regular TAC/RTAC contracts 
specified that at least half of the TAC/RTAC hours allocated to the Nine-Site Initiative be spent on 
site. 

Technical assistance supported under the Nine-Site Initiative differed from and extended the 
regular patterns of TAC/RTAC services in two ways. First, services were directed to Chapter 1 
programs in individual schools rather than to an entire district Chapter 1 project or a state department 
of education. Letters of agreement for TAC/RTAC services are usually between state departments of 
education and the TAC/RTACs. Under the Nine-Site Initiative, the letters of agreement were 
between the TAC/RTACs and the individual schools and/or school districts, although in some cases 
state Chapter 1 officials either reviewed or signed the letters. Second, the initiative was planned and 
announced as a long-term effort, although there was some confusion about the expected duration of 
support. 

Thus, even though the amount of funding for the initiative was relatively low, the 
organization and concentration of services meant that they were more intense than other TAC/RTAC 
services. The long-term relationships with individual schools and districts that ED hoped to foster 
through the initiative represented a new dimension of TAC/RTAC work, 3 1m plans for the Nine- 
Site Initiative reflected a recognition that change takes time, and that while it can be mandated and 
encouraged from above, teachers, principals, and other members of school communities are the ones 
who must carry it out. 



3 The TAC/RTACs had worked in several of the sites prior to the Nine-Site Initiative; however, 
there were no long-term working relationships at the time the initiative began. 



17 




CO 



9 

:RIC 



Findings and Conclusions from the Formative Evaluation of the Nine-Site 

Initiative 



In 1991 , ED contracted with Policy Studici Associatea to conduct a formative evaluation of 
the Nine-Site Initiative. The purpose of that evaluation wai to describe the technical assistance 
services under way at the midpoint of the initiative and to examine their initial impact on Chapter 1 
program improvement efforts in the participating schools, Key findings and conclusions from the 
formative evaluation include the following: 



We found that at the outset of the initiative there was some confusion about the duration of 
HI) support and about expectations for what was to be accomplished. This confusion resulted in a 
lack of commitment in many of the participating schools. Together with the fact that, in most sites, 
participation by the schools was mandatory, this meant that the TAC/RTACs had to spend 
considerable time "selling" the initiative and themselves to the schools. 

Cunfaslon over the duration of support was exacerbated by the fact that TAC/RTAC contracts 
tfftplrad it tha and of the first year of the initiative and TAC/RTACs had to compete for new awards. 
Itt iwn a!!#s, Los Angeles and Mississippi, new contractors replaced the incumbents, which meant that 
HMIiy of tht relationship-building tasks of the first year had to be repeated in the second year. The 
iMIUltluM procaadad fairly smoothly in Mississippi but was more difficult in Los Angeles, where the 
lt«W TAC had to contend with complicated local conditions and the legacy of uncomfortable relations 
HalWten Ihf previous TAC and the district over the Nine-Site Initiative. 



Uial Caatiit f T Iktolal AaaiiUna 

We Identified three factors that defined the local context for technical assistance. First, many 
sites were coping with conditions of extreme disadvantage in their communities. Second, there were 
other, often ambitious state and local reform initiatives under way that took precedence over Chapter 
1 program Improvement. While TAC/RTACs made an effort to coordinate their assistance with state 
and local reform inltiativea in some sites, most notably Kentucky, state and local imperatives often 
overshadowed concerns about Chapter 1 program improvement in the minds of teachers and 
administrators, In a few aites, state and local reform initiatives included technical assistance and staff 
development that made it more difficult for the TAC/RTACs to find a niche for their services. Third, 

5 



20 



Chapter 1 program improvement was not well understood by principals and teachers, many of whom 
felt that the identification process was unfair or misleading. These factors combined to make it 
difficult for the TAC/RTACs to find producth ^ niches in the sites early in the technical assistance 
process. 

TAC/RTAC AtiivUte 

We found that the TAC/RTACs provided a variety of services during the first half of the 
initiative. We also found that the services followed a similar sequence in most of the sites, although 
there was variation in the organization of the activities and the relative amount of time devoted to 
them. In general, TAC/RTACs began by working with principals and others to gain access to the 
schools and establish their credibility as external assisters. Typically, this included completing some 
sort of a needs assessment. In several sites, needs assessments were based on ED's attributes of 
effective compensatory programs or Better Schooling for the Children of Poverty: Alternatives to 
Conventional Wisdom (Knapp & Turnbull, 1990), with results reported to principals or school 
improvement teams. Another early task, completed at ED's direction, was to "clean up" and review 
student test data to determine whether schools had been properly identified for program improvement. 
At least one site carried out an extensive analysis of student test data with district staff, using 
Individual student profiles provided by the testing company to analyze student strengths and 
weaknesses on a multitude of test objectives. In sites that carried out analyses like these, however, an 
examination of test scores generally produced data for schools, not a foundation for the design of 
technical assistance. An analysis of test scores shed some light on student understanding; this 
knowledge informed broader decisions about technical assistance. 

<n several sites, most notably Iowa and Kentucky, initial assistance activities concentrated on 
building and working with school teams. In Iowa, work with the school teams continued as a 
cornerstone of RTAC assistance. In Dade County and Chicago, early plans for assistance and 
program improvement included collaboration between TAC staff and district supervisory staff, with 
local staff responsible for coordinating assistance activities and follow-up to help the schools 
implement new practices. For a variety of reasons, these collaborations did not materialize. 

The primary content focus of technical assistance during the first 18 months of the initiative 
was on classroom strategies for teaching disadvantaged students, particularly in reading and language 
arts. Other topics included helping students develop test-taking skills, increasing parent involvement, 
and coordinating Chapter 1 with regular school programs. Assistance strategies included workshops 
for large and small groups of teachers and principals and one-on-one consultations with district 
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Chapter 1 staff, principals, and sometimes teachers. Teachers and principals who were familiar with 
the assistance spoke highly of the usefulness and quality of the presentations, giving particularly high 
marks to demonstration teaching. One state Chapter 1 director also observed that workshops, the 
traditional fare of staff development programs In many districts, were more, valuable as elements of a 
long-term TAC/RTAC plan and accompanied by follow-up from TAC/RTAC staff directed at 
individual schools or groups of teachers. This arrangement stands in contrast to the common stand- 
alone, isolated events. 

Initial Impact of Technical Assistance Provided Under the Nine-Site Initiative 

In the schools that we visited for the formative evaluation-just over a quarter of those 
receiving assistance-technical assistance provided under the Nine-Site Initiative had contributed to: 
(1) increased awareness of Chapter 1 program improvement requirements and the Chapter 1 
regulatory structure, (2) improved quality of Chapter 1 student test data, and (3) increased familiarity 
with current innovations in the education of disadvantaged students. Not surprisingly, given the 
resources available and the relatively short duration of the assistance, we found little evidence of 
change in instructional approaches, curriculum, or school r r Chapter 1 program organization that 
could be attributed to technical assistance provided under the Nine-Site Initiative. In a few schools, 
however, the assistance did appear to have helped teachers and principals set the stage for potentially 
significant improvements in the future. 

Design of the Third-Year Evaluation of the Nine-Site Initiative 

The purpose of the third-year evaluation of the Nine-Site Initiative was to expand the 
descriptions of technical assistance begun during the formative evaluation, looking more extensively at 
evidence of the effectiveness of the assistance. To that end, the third-year evaluation builds on the 
earlier case studies and expands them to focus more explicitly on the impact of technical assistance on 
(1) individual teachers and administrators, (2) the organization of Chapter 1 programs and schools, 
and (3) student achievement and other student outcomes. We also report on the impact of technical 
assistance on the organization and administration of district Chapter 1 projects, since assistance to 
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district Chapter 1 staff was a central element of technical assistance in four sites: Dade County, 
Kentucky, Mississippi, and the BIA schools. 4 

Sample of Schools Included in the Site Visits 

In each site, we visited two public schools that were receiving assistance under the Nine-Site 
Initiative and a school that had been identified for Chapter 1 program improvement but had not 
participated in the Nine-Site Initiative. We refer to the nonparticipating schools included in our site 
visits as "comparison schools/ 1 Although these schools were not matched to the participating schools 
in any systematic way, our visits to them yielded some insights into the value that TAC/RTAC 
assistance added in the participating schools and increased our understanding of local context. For the 
third-year evaluation, we did not return to the Chapter 1 programs for religious-school students that 
received technical assistance. 

In four sites-the BIA schools, Chicago, Iowa, and Los Angeles-we returned to the schools 
we had visited for the formative evaluation. In four other sites, Chapter 1 administrators 
recommended that one of the original participating schools be replaced; in the ninth site, the RTAC 
recommended replacing one of the participating schools. The comparison schools remained the same 
in all sites. Overall, to conduct the third-year evaluation, we visited approximately one-third of the 
schools that received assistance in the third year of the initiative. All site visits were completed in 
May and June 1993. Follow-up interviews were completed by the end of July. 

The schools that we visited are not necessarily representative of all schools that received 
technical assistance under the Nine-Site Initiative or of all Chapter I schools identified for program 
improvement in the nine sites. During the site visits and ;n the interviews with TAC/RTAC staff, we 
asked district-level administrators and TAC/RTAC staff to comment on the extent to which the 
experiences of the participating schools we visited were typical of all of the participating schools in 
the site. Based on these responses and our analysis of the TAC/RTACs' month-by-month reports, 
which are discussed below, we have two clear impressions about the experiences of the schools we 
visited compared with other participating schools in eight of the nine sites. First, with a few 



4 Under a 1990 Memorandum of Understanding between the BIA and ED, all BIA schools are 
treated as individual districts in ED-sponsored programs. This means that each BIA school has full 
administrative responsibility for the operation of its own Chapter 1 program, including all of the 
paperwork. We learned that the RTAC spent a considerable amount of time assisting schools with 
these responsibilities as part of the school-level assistance services provided under the Nine-Site 
Initiative. 
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exceptions, the schools that we viiited had made lomewhat more progresi than other participating 
schools in the sites. Second, and again with a few exceptions, TAC/RTAC technical assistance 
services had a somewhat more positive impact in these schools. In the ninth site, Los Angeles, the 
TAC had the greatest impact in a school we did not visit, and the impact in that school was reported 
to be substantially larger than in other participating schools in that site. 

Data CflltoUan 

Data collection included visits to state or district Chapter 1 offices and schools in each site, 
interviews with TAC/RTAC staff, and examination of documents. In each school, we conducted at 
least one focus group interview with teachers and interviewed the school principal and other staff, as 
appropriate. The focus group interviews lasted approximately one hour and explored three main 
topics: (1) changes in the school during the past three years, (2) TAC/RTAC technical assistance, 
and (3) the contributions of technical assistance provided under the Nme-Site Initiative and from other 
sources to the changes in the school. Interviews with principals in every school addressed the same 
themes as those with the teachers. In addition to interviewing school staff, we interviewed district 
Chapter 1 staff, including Chapter 1 project directors, other Chapter 1 administrative staff, and, in 
some sites, Chapter 1 supervisors. We also interviewed state Chapter 1 officials who were familiar 
with technical assistance activities in two of the rural sites. On-site data collection was augmented by 
telephone interviews with at least one and usually two TAC/RTAC staff members responsible for 
organizing and providing technical assistance in each of the sites. 

The documents we reviewed included Chapter 1 planning documents from the schools and 
districts, reports of student achievement and other outcomes, letters of agreement, TAC/RTAC 
monthly reports to ED, month-by-month summaries of technical assistance to the schools and school 
district staffs in each site and Education Wee k reports on significant developments in each of the sites 
(e.g., major retorm initiatives, financial problems, union issues, etc.) identified through an on-line 
search. 

The month-by-month summaries of assistance activities were particularly useful in describing 
the quantitative dimensions arid patterns of TAC/RTAC services to all of the school and district 
Chapter 1 projects during the third year of the initiative. At ED's direction and following a standard 
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format, the TAC/RTACs provided the following information about individual technical assistance 
activities: 

• Date, time, and location 

• Topic 

• Assistance strategy/format 

• Number and role of participants 

A limitation of the data included in these reports is that they do not include the amount of time 
TAC/RTAC staff spent preparing for assistance, the amount of time spent traveling, or the amount of 
time spent in telephone conversations with school and district staff. Thus, the descriptions of 
TAC/RTAC services included in Section II of this report underestimate the overall level of effort 
devoted to technical assistance under the Nine-Site Initiative. 



Data Analysis 

Following the site visits, the evaluation team completed case study reports on each of the 
sites. The reports, which are retained for internal project use only, follow a standard guide organized 
around the following general evaluation questions: 

• What technical assistance services were provided in the site? 

• How and to what extent do other s:ate and local education reform initiatives and the 
availability of other technical assistance and staff development activities affect 
Chapter 1 program improvement and technical assistance services provided under the 
Nine-Site Initiative? 

• What changes have occurred in the past three years in the schools visited in this site, 
and how, if at all, has technical assistance provided under the Nine-Site Initiative 
contributed to these changes? 

• Following their experiences in the Nine-Site Initiative, what recommendations do local 
personnel and TAC/RTAC staff have for future technical assistance programs? 

The reports were circulated among the evaluation team for review, comment, and cross-case 
analysis. In looking both within and across the sites, we examined the patterns of assistance and the 
available evidence of changes in the schools. The primary sources of information about the volume 
and content of technical assistance services were the month-by-month summaries, teachers' and 
principals' descriptions of the services and their assessments of the usefulness of the services, and 
TAC/RTAC staff comments. The primary sources of information about changes were the interview 
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responses of teachers and principals and student outcome data, when such data were available to us. 
Responses from district personnel and TAC/RTAC staff also were helpful in understanding the 
changes that occurred in the schools. 

Our general strategy for looking at changes in the schools and the impact of technical 
as ?\ stance was to ask respondents to describe the changes and the impact of the assistance. When 
they identified changes, particularly those they attributed to technical assistance, we invited them to 
provide examples and evidence. This might include sharing samples of new materials they were using 
or descriptions of new instructional strategies they had tried. It also included respondents* accounts 
of changes in the amount and quality of teacher interactions or input in decisionmaking. Our cross- 
case analysis focused on similarities and differences across the sites, particularly in the assistance that 
was provided and the impact it had. 
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EL TECHNICAL ASSISTANCE PROVIDED UNDER THE 
NINE-SITE INITIATIVE 



Thii section describes those features of technical assistance that hive been generally similar 
across the sites and schools, as well as noting important variations within and across the sites. 
Specifically, the section discusses the amount of assistance delivered, who participated in the various 
activities, topics and issues addressed, mode of delivery, and collaboration with other technical 
assistance providers. The data relate primarily to the technical assistance services provided during the 
third year of the Nine-Site Initiative. 

Quantity and Distribution of Services to Individual Schools 

Amount of Trclffliffll A^gtan^ Provided to Schools 

The amount of TAC/RTAC assistance to individual schools during the third year of the Nine- 
Site Initiative varied widely, from 81 hours at a school in Baltimore to 12 hours at a school In 
Chicago. 5 Overall, half of the participating schools received less than 43 hours of assistance, with 
the average number of hours of assistance per school being about 40. One-quarter of the participating 
schools received less than 24 hours of assistance. How do these amounts of service compare to the 
amounts that might otherwise be available? The Chapter 1 Implementation Study (Millsap, Moss, & 
Gamse, 1993) found that the "typical teacher* received an average 23 hours of staff development 
during 1990-91. Thus, for a quarter of the schools participating in the Nine-Site Initiative, the 
amount of assistance available to teachers was less than the average amount of staff development 
normally available to teachers in Chapter 1 schools. Further, the average amount of assistance 
provided during the third year of assistance was only 17 hours more than the average amount of 
assistance available to teachers in Chapter 1 schools in general. Finally, u we discuss in more detail 
below, individual teachers in the participating schools often did not participate in alt of tht technical 
assistance activities, so the amount of services they received is lower than the total amount provided 



5 At sites where the TAC/RTACs delivered services to groups of schools, we counted the 
numbers of hours of on-site assistance to individual schools and the number of hours of on-site 
assistance available to all schools in the total number of hours of assistance to these schools. Our 
estimates of the numbers of hours of technical assistance in several of the BIA schools do not Include 
services provided by consultants hired by the RTAC under the Nine-Site Initiative, RTAC reports did 
not include detailed information about this component of services. 
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In their eehools. Therefore, a significant percentage of participating ichooli received low teveli of 
ftulltince, relative to the amount that might normally be expected; for thete echooli, Nine-Site 
Inlllilive aervicea could hardly he called intensive. 

Much of the variation in the level of service to individual schools can be accounted for by the 
fact that within sites, TAC/RTACs tended to concentrate their services on a few schools. For 
example* among the three participating schools in Baltimore, one school received 81 hours of TAC 
assistance, half of all TAC hours at the site and almost twice as many as the next school. In Detroit, 
the two schools that received the most assistance got twice as many hours as the third participating 
school. In Mississippi, each of the four schools that received the most RTAC assistance received two 
to three times as many hours of RTAC services as any of the other participating schools. 

Across sites, TAC/RTACs that delivered services sitewide or to groups of schools furnished 
participants with more hours of service (albeit in larger groups) than TAC/RTACs in those sites 
where there were no combined activities, with a few exceptions. In Iowa, Mississippi, the BIA 
schools, and Kentucky, where RTACs offered a number of services sitewide or to groups of schools, 
the median number of TAC/RTAC on-site hours available to each school was SO, of which an 
average of 30 hours were devoted to sitewide activities. In sites with almost no sitewide activities, 
TAC/RTACs spent a median of 25.5 hours at each school, with a quarter of the schools receiving less 
than IS hours of service. Sitewide sessions allowed TAC/RTACs to achieve economies of scale and 
deliver training more efficiently. Sitewide activities were effective, however, only when they 
functioned as part of an integrated assistance strategy; this was true in some cases and not in others. 

Many schools received leas Intense service in terms of continuity /er time, as well as in 
terms of total number of TAC/RTAC on-site hours. In Baltimore, Iowa, and Los Angeles, 
TAC/RTACs provided assistance to each Individual school every month of the year. In most other 
sites, however, individual schools did not receive on-site assistance every month. Schools in the BIA, 
Chicago, and Mississippi sites sometimes went three to four months without a visit from TAC/RTAC 
staff. In these sites, some schools received as few as two or three visits from TAC/RTAC staff 
during the year. Again, assistance this irregular cannot be called intensive. 



Mlffibtt of PartldnattiM Schools 

The number of schools that received services under the Nine-Site Initiative declined by one- 
third over the three-year period of ED support. Of the 68 public schools and Chapter 1 programs for 
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religious-school students that began receiving assistance during the 1990-91 school year, 44 received 
assistance during the 1992-93 school year. 6 (See Exhibit 2.) 



Exhibit 2 

School Participation Patterns During the Nine-Site Initiative 
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One school left the initiative at the end of the first year, and 23 schools left at the end of the second 
year or the beginning of the third year, shortly after ED indicated that this would be allowed. While 
no site had fever than five participating schools at the beginning of the initiative, there were three or 
fewer schools receiving assistance in five sites at the end of the initiative. The number of 
participating schools remained the same in two sites. 



f We counted schools in three sites as not participating even though the TAC reports Indicated 
that they were still included in the Nine-Site Initiative. For example, in Dade County, TAC staff met 
with principals and one or two Chapter 1 staff in the two schools we visited; however, according to 
TAC staff* TAC reports, and district Chapter 1 staff, the TAC provided almost no assistance to these 
or other Dade County schools during the 1992-93 school year. Therefore, we counted all Dade 
County schools and the Chspter 1 program for religious-school students as non-participating, In Los 
Angeles, the TAC assisted two schools throughout the year and provided a single workshop in a third 
school The third school was also counted among those not participating. In Chicago, one school 
received three visits from the TAC in 1992-93 to "discuss conditions and needs" and "to assess 
progress/ according to the TAC monthly service summary reports. This school was counted as not 
participating. 
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Schools left the initiative for various reasons. In Baltimore, four participating schools were 
included in a district-supported experiment in which the schools were to be run by a private 
contractor. Inclusion in this experiment precluded participation in the Nine-Site Initiative. According 
to TAC reports, a decision was made to drop a fifth school from the initiative because it had recently 
gone through its third change in building leadership in as many years. One other Baltimore school 
left the initiative at the end of the first year. According to the TAC staff, this school dropped out of 
the Nine-Site Initiative because there were too many other initiatives under way at the same time and 
because the school staff did not think that the TAC had contributed much to their efforts. In Detroit, 
district Chapter 1 officials and the TAC agreed that the TAC would serve only three schools in the 
third year because they concluded that assistance had been spread too thin during the first two years. 
In Dade County, the aftermath of Hurricane Andrew and the presence of other reform initiatives led 
to a decision to focus TAC assistance at the district level instead of at the school level. In Kentucky, 
Chapter 1 officials decided to drop one school because of the school's inability to organize itself to 
receive assistance. In the words of one district administrator, N [Local education agency staff] decided 
not to waste any more money on a bad situation." According to RTAC staff, the second school 
decided to drop out because it was implementing site-based management and was making considerable 
progress without RTAC assistance. 

Teacher Participation Patterns 

Teacher participation in technical assistance activities varied among schools. In Baltimore, 
Iowa, and Los Angeles, technical assistance was targeted to clearly defined groups of school staff: 
third-grade teachers in Baltimore; Chapter 1 and classroom teachers who were members of the 
"IRISE" team in Iowa; and teacher aides in Los Angeles. Membership in the target groups remained 
generally stable, with the result that essentially the same staff attended TAC/RTAC assistance 
activities every month. This pattern ensured that a relatively small subset of school staff received 
relatively high levels of assistance. At other sites, teacher participation was more broadly distributed, 
but alio mora uneven, TAC/RTACs targeted different groups (aides, Chapter 1 teachers, upper 
•lemantary reading teachers, etc.) for different assistance activities. While larger numbers of staff 
had an opportunity to work with the TAC/RTAC at one time or another, individual teachers typically 
received much leii assistance than was available to the school as a whole. In some schools, 
TAC/RTAC activities were open to all interested teachers, so that the most enthusiastic teachers 
received the most assistance. In o*her schools, decisions about which teachers would participate in 
technical assistance activities reflected little planning. As one indicator of teacher contact with the 
TAC/RTAC In schools outside Iowa, Baltimore, and Los Angeles, teachers participating in focus 
group Interviews (who tended to be those most involved with the TAC/RTAC) said that they attended 
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a TAC/RTAC activity or met with a TAC/RTAC staff member two to four times in the last year. In 
some cases, these activities took place several months apart. 

Inconsistent participation patterns and lack of communication to teachers about TAC/RTAC 
activities produced frustration at some schools. At one rural school, there were complaints of staff 
development opportunities being "spotty ," with teachers often unaware of upcoming events until the 
last minute and therefore finding it inconvenient to attend. In another rural site, one teacher 
expressed her frustration at not knowing more about the various technical assistance activities; she 
said that the only reason she knew anything about RTAC assistance was that she asked other teachers 
what they were doing with the RTAC. 

In some schools, TAC/RTAC staff spent limited amounts of time with teachers. TAC/RTAC 
assistance with parent involvement programs in two schools, for example, involved only one or two 
teachers. In one urban site, TAC staff worked with the school-community liaison and principal; at 
another school, also in an urban site, the TAC staff member served as a consultant to the head of the 
parents' organization. In other schools, TAC/RTAC staff worked primarily as consultants to the 
principal, providing information on topics the principal wanted to know more about or helping solve 
problems in planning, assessment, and staftipg. 

Content of Technical Assistance 

In all sites, the content of technical assistance usually included: (1) training in new 
instructional practices, especially "hot topics" like cooperative learning, whole language, and math 
manipulates; (2) assistance with Chapter 1 program administration and the interpretation and use of 
student test data; and (3) assistance or training related to local or state initiatives, from school-based 
parent involvement programs to district curriculum initiatives to new statewide assessment programs. 

An analysis of the number of hours TAC/RTACs spent addressing various assistance topics 
shows only a few sites with a clear content focus over the course of the third year. In Baltimore, a 
little over half of TAC time on site was devoted to topics the TAC declared were central to technical 
assistance in the site: cooperative learning, teaching advanced skills and problem-solving in math, 
and preparing students for the Maryland Student Performance Assessment Program. In Iowa, 40 
percent of RTAC time was devoted to professional development in integrated literature-based 
language arts instruction. In Los Angeles, 55 percent of TAC time was devoted to staff development 
on teaching reading through literature in bilingual or ESL programs. 
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In other sites, TAC/RTACs addressed several different topics simultaneously; a typical 
TAC/RTAC agenda might center on Introducing whole language teaching practices to teachers and 
include working with a principal or program improvement committee to interpret test data or develop 
a new Chapter 1 program improvement plan as well as dealing with other miscellaneous topics 
requested by teachers. In some sites, technical assistance appeared to have no focus at all. 
TAC/RTAC monthly reports typically mentioned a long list of topics, with very little time devoted to 
each. In one urban site, for example, TAC reports listed the following topics, with less than 10 
percent of TAC time devoted to each: coordination of Chapter 1 and the regular program, team 
building, tutoring programs, reading/writing/whole language, general instruction, parent involvement, 
Chapter 1 program improvement and evaluation, test-taking skills, cooperative learning, staff 
development plans, math and math manipulates , team teaching, extended-day programs, student 
behavior, and authentic assessment. 



Instructional Practices 

TAC/RTACs differed in their approach to the introduction of new instructional practices. 
The most ambitious TAC/RTAC efforts to introduce innovations in classroom instruction attempted to 
ground teachers in the research and rationales underlying whole language, literature-based language 
arts, or the teaching of problem solving and other advanced skills in mathematics. This assistance 
aimed to help teachers use new materials and activities in their classrooms, integrate them with other 
activities, and understand and articulate why they were valuable. In Iowa, for example, the primary 
focus over three years was on developing a literature-based language arts program and integrating 
language arts across the curriculum. During a focus group interview, teachers who experimented 
with literature-based language arts activities in their classrooms were able to explain and debate the 
merits of children's literature versus basal readers for the teaching of reading. This pattern of 
technical assistance attempted to alter teachers 1 beliefs about how students learn and, subsequently, 
their entire approach to teaching. Evidence from our site visits, particularly the focus group 
interviews, indicates at least some success for these attempts. 

In other sites, assistance strategies had a more limited goal of introducing information about 
discrete instructional practices that teachers could use immediately. In experimenting with these 
activities, teachers added to their teaching repertoires but did not develop a clear understanding of the 
rationale for the new activities and were unable (or disinclined) to integrate these activities with other 
things they did in the classroom. For example, teachers in one rural site described RTAC 
demonstration lessons with great enthusiasm but were hard-pressed to explain how these lessons might 
influence their teaching in general. One RTAC demonstration lesson was an extension activity for the 
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children's book Knots on a Counting Rope, in which a Native American child discovert the story of 
his b*rth from conversations with his grandfather about a counting rope. In the extension activity 
introduced by the RTAC, students interviewed their parents and other relatives at home about the 
story of their own births and then presented these stories in class, creating their own class counting 
rope. The activity provided an opportunity for teachers to take their students' life experience as a 
starting point for learning, a basic precept of whole language teaching. While teachers were 
impressed with how their students enjoyed the activity and planned to use it the following year, they 
were unclear about how they could use similar activities in the rest of their teaching. What they 
talked about was repeating the activity, not expanding on it. 

Chapter 1 Administration. Evaluation, and Data Collection 

In many sites, a fair amount of technical assistance was devoted to working with principals, 
Chapter 1 teachers and administrators, and district administrators on topics including Chapter 1 
program evaluation, coordination with the regular program or other state/local initiatives, and 
interpretation of test data. 

TAC/RTAC staff began their work in many sites by helping administrators interpret test data 
and make decisions about service delivery. They built on this traditional TAC/RTAC function in 
some sites by adapting their evaluation expertise to local needs. For example, RTAC assistance to the 
district office in Kentucky focused on developing a set of measurable desired outcomes to evaluate 
Chapter 1 programs under the state's initiative to substitute new state assessments for the 
Comprehensive Test of Basic Skills in Chapter 1 evaluation. During 1992-93, an interim year in the 
transition to a new state assessment system, districts were to use locally developed desired outcomes 
to determine Chapter 1 program improvement status. The RTAC helped the district develop the 
outcomes and design behavioral checklists that classroom teachers would complete to record progress 
toward the desired outcomes. In a similar project in Mississippi, RTAC assistance included help in 
designing districtwide instructional management objectives that are coordinated with the Chapter 1 
program. The RTAC then helped the district develop "maatery checklists" for teachers to use in the 
classroom. The RTAC was instrumental in selling the use of these checklists to teachers and reluctant 
administrators. 

TAC/RTAC stiff also helped district staff with other initiatives. In Dade County, where the 
TAC primarily assisted the district office, the TAC helped the district prepare proposals for parent 
involvement and computer-assisted instruction programs, and helped plan their implementation. 
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In sites where TAC/RTACs helped teachers and administrators examine their program' 
plan improvement efforts, school staff appreciated the expertise and perspective TAC/RTAC 
brought as outside consultants. In addition to helping schools use test data and evaluation information 
to design sound programs, TAC/RTACs provided information on other programs and objectivity. As 
one urban principal noted, the TAC provided examples of what other schools were doing, "kept us 
clear on the regulations, and helped with understanding the statistics. It is helpful to have someone 
who is not in the school [to bring perspective]/ (n a rural site, school and district administrators said 
it was especially valuable to work with an outside expert who had no monitoring responsibilities. 
Because the RTAC did not have to enforce compliance with Chapter 1 regulations, Chapter 1 
administrators could discuss problems openly without fear of sanctions. As a result, according to one 
district Chapter 1 director, H I can tell [the RTAC] things. I don't feel like I'm airing my dirty 
laundry. I can tell [the RTAC] about my insecurities." As an RTAC staff member explained, M When 
you want to improve, but don't know how, it's much easier to go to someone like us. You're not so 
comfortable discussing your problems with the superintendent, but you can admit to an outsider that 
you need help! M 



Schoolwide Protects 

Some TAC/RTAC assistance included help in developing sdioolwide projects, an endeavor on 
the minds of staff in many of the schools we visited. Some were preparing their evaluations as their 
three-year school wide project was ending. Others had recently become school wide projects and 
needed help in developing new service delivery options. In several schools, TAC/RTAC staff 
consulted on schoolwide project plans or third-year evaluation designs. In Mississippi, the RTAC 
helped launch a schoolwide project by enlisting teachers from an established schoolwide project to 
make presentations to reluctant teachers at a school that had just gone schoolwide. In Detroit, TAC 
staff arranged for district Chapter 1 administrators to attend a national schoolwide project conference. 



State and Local Reform* 

In most lites, TAC/RTAC assistance attempted to strengthen school-level efforts to meet the 
requirements of state and local reforms. For example, Chapter 1 program improvement in Baltimore 
was integrated into the district's own school improvement initiative. The topics addressed through the 
Nine-Site Initiative in Baltimore-the use of manipulates for concept development, problem-solving 
strategies, advanced applications of mathematics concepts, and cooperative learning-were closely 
aligned with the curriculum and assessment priorities that the district had emphasized over the past 
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few years. In addition, a number of TAC activities specifically addressed the Maryland State 
Performance Assessment Program, which, among a long list of other requirements, required students 
to solve some problems in cooperative groups. 

RTAC assistance in Kentucky provided the best example of assistance geared to state-level 
reforms. A key theme of early assistance in Kentucky was to examine the potential overlap between 
the mandates of KERA and Chapter 1 program improvement requirements. Chapter 1 program 
improvement was not a priority for either teachers or principals, but understanding and responding to 
KERA mandates were. By showing the relationship between the two sets of requirements, the RTAC 
provided a valuable service, helped schools see a way through a dense thicket of reform initiatives, 
and established itself as a credible source of assistance. Addressing KERA mandates came to be a 
dominant them* in RTAC assistance in Kentucky; assistance in the third year included an ongoing 
review of KERA assessment requirements, portfolio assessments in mathematics, and a variety of 
issues related to instruction and assessment in writing. 

Technical assistance in the BIA schools addressed some local concerns that included and 
surpassed Chapter 1 . The BIA had encouraged schools to seek North Central Accreditation, and the 
RTAC provided training and Assistance in the application process. Also, alternative assessments were 
extremely popular throughout the region, given the historically low performance of Native American 
children on norm-referenced testa. One of the schools we visited was pursuing portfolio assessment 
as an alternative to norm-referenced tests for showing what students know and can do; alternative 
assessment was a focus of RTAC assistance during the Nine-Site Initiative. 

Planning 

Planning for Nine-Site Initiative activities took up an average of 20 percent of TAC/RTAC 
on-site time across all sites. In many sites, planning activities ranked first or second among topics for 
technical assistance in the number of on-site hours devoted to them. In some sites, time devoted to 
planning appears to have been a necessary preparation for the technical assistance activities that were 
to follow. For example, in Iowa the RTAC devoted a considerable amount of on-site time to 
planning activities (29 percent of all on-site hours), but an examination of month-by-month reports 
shows that these activities were a fundamental part of a long-term technical assistance strategy that 
hinged on carefully tracking technical assistance in order to maintain a focus and adhere to a strategic 
plan. In several other sites, what were identified as planning sessions appeared to have had the more 
limited purpose of deciding what to do in the immediate future without much attention to what had 
already been done or reasons to choose a particular activity. In one urban site, more than half the 
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TAC on-site time was devoted to Nine-Site Initiative planning and needs assessment, leaving little 
time for other kinds of assistance. The fact that so much time was devoted to planning in the third 
year of the initiative probably indicates that TAC/RTAC services were not an integral part of ongoing 
school improvement efforts or that no such efforts were made. 



Strategies for Organizing and Delivering Technical Assistance 

In several sites, TAC/RTACs had clearly identifiable strategies for delivering technical 
assistance. In sites where TAC/RTACs used cross-site activities or activities involving groups of 
schools, they achieved some economies of scale by presenting workshops to large groups that 
included participants from a number of participating schools, thereby providing more contact hours to 
schools and teachers than visits to individual schools alone would allow. Iowa, Mississippi, and the 
BIA are good examples of sites where TAC/RTACs used this strategy. 

In Baltimore, the TAC pursued a different but equally well-defined strategy. TAC staff there 
spent almost half their on-site time-a very high concentration relative to other sites-in classroom 
observation and demonstration teaching in the third year. Combined with consultations with 
individual teachers outside of class, this one-on-one assistance took up almost two-thirds of the TAC's 
time on site. Los Angeles represented a combination of these two strategies: according to monthly 
reports and interviews with TAC staff, assistance in one of the participating schools in Los Angeles 
included a series of large group workshops closely followed by one-on-one sessions in classrooms that 
included observations and consultation. 

Technical assistance in Iowa reflected the most thoroughly developed strategy for organizing 
and providing technical assistance. In Iowa, the same people participated in the technical assistance 
for all three years. When the Nine-Site Initiative first started, the RTAC required each school to send 
a team to the kick-off meeting in Ottumwa; the team was required to include, at a minimum, the 
principal, Chapter 1 teachers, and classroom teachers. The RTAC focused all technical assistance on 
activities that would bolster the teams* capacity to work together and support school improvement 
efforts-team planning sessions, team attendance at workshops, team consultation about school-specific 
issues and strategies. Teams did not do anything beyond attending RTAC activities together-they did 
not take on management functions or other governance roles within the school per se. However, their 
participation in RTAC activities-as a group-ensured attendance by the same staff at each activity and 
created a cohort of teachers at the school who shared a knowledge base. 
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With the exception of Baltimore, in sites where one-on-one sessions were the dominant 
TAC/RTAC strategy, sessions tended to be discrete, isolated events with no relation to one another; 
they did not necessarily follow from careful diagnosis, nor build on previous activities. In some 
schools where one-on-one sessions were the dominant mode of assistance, only a few teachers even 
knew about the assistance, and it was in no way a part of the institutionalized routines of the school. 
Scheduling concerns in some cases played a major role in shaping the pattern of assistance; where 
group activities were difficult to schedule, activities tended to involve only one or two teachers. Even 
these small-scale activities were sometimes difficult to fit in; teachers in one rural school said that 
while they knew in advance which days the RTAC would visit their classes, they did not know what 
time RTAC staff would arrive or how many lesions they planned to observe. In contrast, principals 
in the Iowa schools we visited planned in advance to have substitutes or school staff not on the RTAC 
team cover classes to release the team teachers for RTAC activities. RTAC staff reported that 
substitutes were available to cover classes during RTAC workshops and training in Mississippi. 



Cumulative Versus Reactive Assistance 



In sites where a clear content focus and a well-defined strategy for delivering services guided 
the technical assistance process, the assistance was much more likely to be cumulative over the three 
years of the initiative. In sites without these elements, technical assistance tended to be tentative, 
even random. In several sites, the TAC/RTAC itself brought a well-developed assistance plan to the 
schools and, following some initial successes, persuaded the schools to "buy into M the approach. 
Once again, technical assistance in Iowa was the best example. The RTAC approach, with the 
support of the state Chapter 1 coordinator, was to encourage and assist the schools in forming teams 
and to focus the content of assistance on literature-based language arts programs. In Mississippi, 
following RTAC observations over several visits, the RTAC convinced one school that assistance 
should focus on the instructional aides. In addition, RTAC staff organized a number of technical 
assistance activities to address problems they personally had identified on visits to schools: very 
dirty, drab buildings; apathy among students and teachers; and lack of coordination in staff 
development, among other things. These examples suggest that while questions and requests from 
school staff were important ingredients in setting the agendas for assistance, the TAC/RTACs were 
more likely to be the sources for clear, long-term plans for cumulative technical assistance. 

In sites where there was no clearly defined agenda for assistance, TAC/RTAC services 
appeared to be mostly reactive, as TAC/RTAC staff attempted to respond to requests from all 
quarters. In these sites, most technical assistance activities addressed questions raised by teachers. 
Principals or Chapter 1 teachers conveyed teacher requests to the TAC/RTAC before scheduled visits, 
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and the staff prepared sessions that targeted specific requests and inquiries. One Chapter 1 teacher 
who helped organize RTAC visits to a rural school commented: 

[RTAC staff] always found out what we needed most. . . . Some teachers wanted to observe 
[a] reading [lesson taught by the RTAC staff], some wanted help with writing or math 
portfolios. When [the RTAC staff member] went to the classroom, if [the teacher wanted] 
writing, math, [or] a reading class, whatever, she did it. 

Not surprisingly, teachers appreciated this highly individualized assistance, which they 
perceived as responsive to their expressed needs. On the other hand, several administrator*) we spoke 
to saw a problem with this approach. One called it N spray and pray/ implying that the technical 
assistance lacked a guiding framework. A state Chapter 1 administrator observed: 

[There should be] an underlying theme. You can't talk about authentic assessment one day 
and team teaching the next and hope something catches. You need a focus for what you're 
working towards. 

Assistance topics at the two schools we visited in an urban site illustrated how assistance 
activities in some places lacked continuity over time. In one school, the first meeting of 1992-93 
focused on ways to develop and use the school's new management teams. The next assistance 
activity, which did not occur until mid-January, included team-building workshops for the 
management teams. The month-by-month reports mention no further work with the teams, Instead, 
the February sessions, which were the final ones of the year, were spent working with groups of 
aides and career service staff. In the second school, the first visit by the TAC, which took place in 
January 1993, resulted in a decision to have monthly, voluntary afterschool "shop talk sessions." The 
first session was held the following day. TAC staff did not visit this school again until June for a 
session that was intended to "review initial report and observe and note progress in regard to report," 

To be sure, technical assistance activities that respond to teacher concerns and requests, no 
matter what those requests are, play an important role in the best-designed technical assistance 
initiatives. In Iowa, one of the RTACs goals for the Nine-Site Initiative was to build the capacity of 
teams from each school to identify their own needs. For example, when a team decided to explore 
math manipulatives, the RTAC was quick to deliver information and materials. According to an 
RTAC staff member working in Mississippi, workshops that teachers found immediately applicable to 
their classrooms, no matter what the topic, played an important role in developing the trust of 
teachers. Her first workshops were designed to give teachers suggestions they could use immediately 
in class, keeping teachers within their comfort zone: 
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If 1 show them something they c»n du, then they'll change. . . . Because what I gave them in 
the beginning was so eaiy, they listened to me when I gave suggestions. It's an important 
motivator if they know I care and they know they will get something out of it. 

At some sites, TAC/RTACs had difficulty settling into a comfortable relationship with schools 
and consequently were slower to develop coherent, cumulative assistance programs. In one urban 
site, for example, the TAC floundered to some extent as it tried to launch technical assistance under 
the Nine-Site Initiative. Some schools did not find the assistance useful compared with other 
resources available to them. Others found unsatisfactory the proposed plan for delivering the 
assistance to all teachers in a central location (requiring lots of teacher travel time). In some schools, 
personality conflicts blocked the success of assistance efforts. As different strategies failed to achieve 
positive results, the TAC would develop a new plan. It appears that this was less a conscious trial- 
and-error system, through which the TAC made a concerted effort to learn from its mistakes, and 
more an ongoing attempt to fulfill contractual obligations while responding to immediate, expressed 
requests from the schools. 

Collaboration and Coordination 

One way TAC/RTACs can extend the impact of their activities is to collaborate with other 
assistance providers serving schools, including district staff. Coordination and collaboration among 
assistance providers has the potential to marshal! more assistance resources for schools and allow 
schools to align different, and sometimes competing, assistance agendas. However, collaboration and 
coordination also require an Investment of time and resources that made it difficult for TAC/RTACs, 
Working Within limited budgets, to develop collaborative relationships with other technical assistance 
providers. As a result, we found few examples of TAC/RTAC collaboration with other external 
ssslsistuie providers, 

In Detroit, TAC staff made an effort to work closely with area administrators 7 in the third 
year of the Initiative In order to increase the impact of the technical assistance. Area administrators 
participated in a general meeting at the beginning of the year and formed teams to support technical 
assistance at the two participating public schools* TAC staff worked with area-level administrators, 
encouraging and advising them on their activities in participating schools and providing them with 
current research on Chapter 1 , Area administrators attended some of the TAC workshops for 



7 The Detroit school district is divided into six administrative areas, each with its own Chapter 1 
office, curriculum director, and teacher specialists. 
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teachers and joined them in meetings at the schools. At one school, they plan to replace TAC staff 
with a person from the area office who will provide support for the program improvement teams at 
the school in 1993-94. Although these collaborative efforts have succeeded in bringing district 
administrators into the schools more often, area administrators noted that they have had difficulty 
fulfilling all the obligations assigned to them due to their other responsibilities. 

In the BIA schools, the RTAC tried to continue targeting assistance to the schools beyond the 
1992-93 school year by coordinating with one local education agency. The RTAC included local 
agency staff in several planning and technical assistance training sessions during the 1992-93 school 
year in order to help the school and agency staff link the district's school improvement efforts (e.g., 
new curriculum, use of math manipulatives) to those of individual RTAC -assisted schools. In 
Mississippi, administrators in both the districts we visited said that the RTAC played an active role in 
coordinating its efforts with other district staff development activities; many times the RTAC did this 
by suggesting staff development activities that complemented RTAC services. 

At other sites, attempts to collaborate with district administrators in delivering technical 
assistance produced fewer tangible results. In Dade County and Chicago, initial plans for technical 
assistance called for supervisors from the district's regional offices to work with the TAC to plan and 
conduct follow-up activities and coordinate technical assistance with other staff development activities; 
however, these activities never happened. In the case of Chicago, staff reassignments necessitated by 
budget shortfalls prevented administrators from being as involved as they had planned. Supervisors in 
Dade County said they had only limited involvement with technical assistance activities and that much 
of their time was spent monitoring compliance with Chapter 1 requirements. In reflecting on TAC 
experiences in one site, a TAC staff member concluded that: 

District support [for a variety of programs] is not always positive. It may not always fit in 
and it may be a day late. I have not spent a lot of time with district staff and I am not sure 
what they know [about what we are doing]. 

In one urban site, TAC staff concentrated their limited resources on working almost 
exclusively with teachers instead of attempting to build a collaborative relationship with the district: 

We never set out to draw a schematic of all the forces that affect the schools we worked with 
in [the site]. We simply didn't have the resources and never even considered it. We sought 
to narrow our focus, not expand it* 

In Mississippi, the RTAC worked actively to coordinate its activities with those of technical 
assistance providers outside the district. These efforts at collaboration were as important as the 
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assistance the RTAC provided directly to schools. The RTAC staff in Mississippi were instrumental 
in seeing that teiehert from at least one district in the site attended a number of whole language 
seminars, such as the Wright Group whole language workshop, and professional meetings, with 
funding from the district's Chapter 1 program. While the district sent teachers to these activities 
Independent of RTAC assistance, the Chapter 1 director credits the RTAC with Introducing the 
concept of whole language teaching in the first place and convincing the district that teachers needed 
additional professional development. We heard many times in Mississippi about various workshops 
that teachers had attended, paid for by the district Chapter 1 program, that the "RTAC knew about" 
or the "RTAC sent us the information on." Usually these workshops directly complemented the 
RTAC agenda. For example, all principals in one district attended a workshop on "Inviting Schools," 
suggested by the RTAC because of the RTAC's concern about cleanliness in the districts' schools, 
although this workshop was not formally sponsored or paid for by the RTAC. RTAC staff also 
introduced a number of new programs that districts later adopted, including "Books and Beyond" and 
several NDN programs. In order to help school districts find the money to train teachers, the RTAC 
worked with district staff to coordinate Chapter 1, Chapter 2, and Elsenhower funds. While the 
amount of additional professional development these efforts made possible was relatively small, one 
district administrator told us that the fact that it waa coordinated with other activities sponsored by the 
district multiplied its effects. 
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III. CHANGES IN SCHOOLS AND OUTCOMES OF TECHNICAL 

ASSISTANCE 



Dim from thft end of tht third year of the initiative, together with a review of the data 
collected during the second year ai part of the formative evaluation, revealed changes related to 
school improvement in most of the participating schools we visited. However, the magnitude and 
scope of these changes varied widely. While a few schools experienced significant changes in 
organization* climate, or instruction* many made only modest progress in improving their 
instructional programs. In addition, a number of the changes were clearly due to factors other than 
the technical assistance (e.g., state or local reform mandates, other external assistance, or self- 
initiated programmatic adjustments) 7 1 some schools the pace of change was slowed by these and 
other factors; a lack of progress cannot always be interpreted as a sign that the technical assistance 
did not achieve its goals. In this chapter, we discuss the changes we observed in the schools we 
visited during the third year of the Nine-Site Initiative. These changes occurred in five broad areas: 
(1) Chapter 1 service delivery and program design; (2) teachers and school-level administrators; (3) 
instruction and assessment; (4) student outcomes; and (5) state and district administration of 
Chapter 1. 

Several factors affect analysis of the outcomes of the Nine-Site Initiative. First, given the 
complexity of school change and the relatively short time span of the Nine-Site Initiative, limited 
resources and unclear goals lessened the potential impact of TAC/RTAC assistance from the outset. 
Second, participants in the initiative had different sets of expectations that translate into different 
definitions of success. For example, some of the TAC/RTACs, in their own evaluations of the 
initiative, list as a major accomplishment gaining access to schools and getting the attention of school 
staff. This represents a much less ambitious agenda than, for instance, facilitating the creation of 
teacher networks that can generate reform agendas and activities on their own, which the research 
suggests is a desirable outcome of external assistance (Hargreaves & Fullan, 1992). Some principals 
also expected the technical assistance to attack low test scores, the trigger for identification for 
program improvement, directly by working with teachers on test-taking strategies. 

A third, and perhaps the most important, mitigating factor is the effect of local context. Just 
as technical assistance affects school and district organization and events, it is also shaped by them, 
Indeed, the local context not only helps define needs but also defines opportunities for and constraints 
on external assistance. It is unfair and unhelpful to compare outcomes across schools and across sites 
without taking into account the dramatic differences in local contexts and the variations in the initial 
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circumstances into which technical assistance was introduced. It is well beyond the scope of the 
evaluation of the Nine-Site Initiative and this report to describe fully all the local circumstances that 
buffet the schools that received assistance under the Nine-Site Initiative. However, as examples of 
factors that influence what happens in schools, consider the following: 



• In Chicago, the school reform initiative, with its emphasis on transferring control of 
schools away from the Chicago Public Schools, has politicized many communities and 
focused attention on school governance. Early reports on the reforms suggested that 
attention to governance deflected attention from efforts to improve curriculum and 
instruction. More recently, it has been reported that while the reforms have yet to 
affect classrooms in many schools, they have begun to make a difference in some. 
However, even as this news was announced late in the 1992-93 school year, state and 
local budget analysts were predicting that enormous budget shortfalls would make it 
extremely difficult, if not impossible, for schools to open in the fall unless there were 
major budget cuts and possibly layoffs. 

• In Los Angeles, teachers 9 concerns about a budget shortfall and the prospect of layoffs 
and salary reductions led them to take four strike votes in 1992-93. District financial 
problems were solved, at least temporarily, by some last-minute maneuvering in the 
state legislators. Soon after this crisis eased, a report that was highly critical of the 
organization find management of the district was released amidst a flurry of charges 
and countercharges of serious mismanagement. Among other things, the report called 
for a complete restructuring of the governance of the nation's second largest school 
system. 

• In September 1992, Hurricane Andrew swept through Dade County and left a trail of 
destruction that took its toll throughout the entire school year. In the schools included 
in the site visit, which were spared serious damage, the aftermath of th<; storm created 
almost unbearable stress for teachers. They had to cope with students' fears about the 
storm and the havoc it wreaked on their lives. In addition, many teachers faced 
serious problems of their own as they coped with the tasks of relocating, rebuilding, 
and, by the end of the school year when repairs were completed, moving back into 
their homes. According to the principals, a district decision to eliminate teacher 
professional development days as a way of coping with the shortened school year 
added to the problem by taking away opportunities for "teachers to relax and recharge 
their batteries/ 

• In Detroit, a teachers' strike delayed the opening of school by five weeks. The 
disruption made it difficult to initiate technical assistance activities during the fall. 
Problems continued as teachers struggled with trying to make up for lost time by 
working longer hours. In one of the participating schools, the strike exacerbated 
existing tensions among the staff and resulted in even lower staff morale. 

• While all of the schools included in the Nine-Site Initiative must cope with the effects 
of poverty, some of the schools included in the visits to Mississippi, Los Angeles, and 
Chicago face particularly acute problems that stem from extremely high concentrations 
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of poverty, students' lack of proper nutrition, inadequate health care, and the almoit 
complete absence of support from families and communities. 

All of the sites posed formidable challenges to the TAC/RTACs; however, the urban sites, in 
general, appear to have been more difficult places in which to work with individual schools. A 
shorthand way of characterizing the difference is to say that there is more "noise H in the urban sites 
than in the rural sites. Nearly all of the accounts of extenuating circumstances we heard-teachers' 
strikes, budget shortfalls, and complicated bureaucracies-we heard in cities. From the beginning, the 
TACs had to negotiate large, complex urban school bureaucracies for which the Nine-Site Initiative 
was but one more program competing with myriad other programs that were already under way, 
although not always in the schools slated to participate in the Nine-Site Initiative. In the rural sites, 
there were fewer competing programs and other conflicts affecting the schools. It was also easier, for 
reasons that we do not fully understand, to gather teachers from all or most of the schools in the rural 
sites and to take advantage of some economies of scale in providing assistance. On balance, district- 
level support and advocacy for the assistance provided under the Nine-Site Initiative also seemed 
stronger in the rural sites. 

Our analysis of outcomes is based on a snapshot study. We do not have enough information 
to precisely pinpoint the changes that took place in the schools v/e visited over time, nor can we 
accurately predict what will occur next year. 

Chapter 1 Programs 

Our findings from the formative evaluation ot the Nine-Site Initiative, along with the results 
from several larger studies of the implementation of the program improvemet provisions of the 1988 
Hawkins-Stafford Amendments (Millsap, Moss, & Gamse, 1993), lead to the conclusion that 
identification for program improvement is rarely the driving force behind meaningful school 
improvement. In fact, in many schools, it does not even inspire much tinkering with the Chapter 1 
program beyond writing a new plan. Our evidence from the participating and comparison schools 
included in our evaluation supports this conclusion; although we saw some schools making serious 
efforts to improve, these efforts most often came about for reasons other than identification for 
program improvement. 

When we first visited the nine sites, we fend that designation for Chapter 1 program 
improvement had caused a degree of consternate .*«ong some principals and teachers. Some were 
simply conftised by the new accountability system and what they were required to do. Most 
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principal! and teachers understood that the problem wu related to low tut scores and that the obvious 
remedy was to H gtt the scores up/ Requirements to prepare and Implement program improvement 
plans generally added more work and Increased the leveli of confaaion and frustration. A more 
fundamental problem wu that principals and teachers In the schools we visited, including the 
comparison schools, tended not to trust the identification process, and consequently did not see 
addressing the Chapter 1 program improvement requirements as a high priority or a schoolwide 
responsibility. 

If anything, we detected less concern about complying with Chapter 1 program improvement 
requirements at the end of the 1992-93 school year than at the time of the first round of site visits. In 
general, the principals and teachers with whom we spoke were concerned about the quality of their 
school programs. However, their overall lack of understanding of the Chapter 1 program 
improvement requirements, their inexperience with using Chapter 1 student achievement data to 
inform decisions about improving curriculum and instruction, the absence of any serious rewards or 
sanctions, and the almost complete lack of serious sustained support and follow-up by state and local 
Chapter 1 administrators make Chapter 1 program improvement requirements a hollow bureaucratic 
shell in the professional lives of principals and teachers. Further, most schools do not have the 
expertise to examine their programs critically, the flexibility to redirect existing resources or tap into 
new ones, or the information they need to make serious changes in instruction or school organization, 
Indeed, these are the very weaknesses that school-level assistance is often designed to address. 

To the extent that a program of technical assistance can help foster substantive change, an 
important first step ia to address these capacity issues. An examination of the Chapter 1 program is a 
logical starting point. This section examines the effects of the technical assistance activities on the 
organization of Chapter 1 programs, including design and service delivery, as the TAC/RTACs 
worked to enhance the capacity of the schools to improve. 

IntEMMd Knowledge About Chapter 1 Requirement! 

Our evidence indicates that, in many of the schools that received technical assistance during 
the third year of the initiative i teachers and administrators know more about Chapter 1 requirements, 
regulations, and options now than they did three years ago. Staff in almost all of the study schools 
credited the TAC/RTACs with helping them better understand program improvement, which has 
proven a somewhat elusive concept in many schools since the enactment of the 1988 Hawkins- 
Stafford Amendments. The TAC/RTACs served aa an important resource for sites on Chapter 1 
regulations; however, bringing district staff up to speed on compliance issues was not the focus of 
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assistance in any site. Instead, this kind of assistance served as a necessary first step toward program 
improvement. In Iowa, for example, the schools we visited reported that they had previously 
understood coordination between Chapter 1 and regular program teachers regarding lesson plans to be 
illegal; the Chapter 1 programs in these schools were cut off from contact with the regular program. 
Guidance from the state Chapter 1 coordinator and RTAC staff on new Chapter 1 program designs 
and the regulations governing them encouraged teachers to consider alternate program designs and 
integrate Chapter 1 into the regular school program. 

Most of the schools we visited were in joint program improvement planning with the state 
education agency, because the law requires that the state must intervene in schools that do not 
demonstrate improvement after two years of identification. However, in no site did we see evidence 
of sustained joint planning. Some schools were unaware of this requirement; others reported 
attendance at one meeting related to continued identification but no follow-up. In addition, we found 
no evidence of monitoring of program improvement activities, although there is considerable attention 
to test results as determinants of whether or not a school is designated for program improvement. 
Principals with whom we spoke reported little or no feedback on Chapter 1 program improvement 
plans. 

In the rural sites, schools reported that they lacked basic information about Chapter 1 prior to 
their exposure to the RTACs; staff were either misinformed about regulations or unaware of 
opportunities (e.g., schoolwide projects) for which they were eligible. In these places, teachers and 
administrators found the RTAC to be a valuable resource as an accessible source of accurate 
information abou; Chapter 1 regulations and the options for program design and evaluation. For 
example, the BIA schools had recently taken on responsibility for their Chapter 1 programs; they do 
all the paperwork and must deal with federal regulations that the agencies used to handle. Staff in 
these schools used the RTAC as a resource as they learned the basics of running a Chapter 1 
program, from filling out an application to testing students for eligibility. In Kentucky, the RTACs 
work with the district was an important contribution as the district embarked on the process of 
internalizing and implementing KERA. In the individual schools, the enactment of KERA generated 
confusion as schools struggled to understand the myriad requirements of the new law. During the 
Nine-Site Initiative's first year, the RTAC helped the schools and district explore the overlap between 
KERA and Chapter 1 program improvement requirements. 

In contrast to these rural schools, schools in the urban sites generally hid access to 
information about basic regulations and evaluation requirements. Large urban districts generally have 
a sizable Chapter 1 staff that is responsible for keeping schools in compliance and making sure they 
have the opportunity to explore allowable new options. For example, in Detroit, where both 
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participating public schools were schoolwide projects, district Chapter 1 staff worked with school staff 
to apply for ichoolwide itatui and design acceptable plans for implementation. The Michigan state 
Chapter 1 staff enthusiastically encouraged all eligible schools to become schoolwide projects and 
supported district staff with information and assistance. Although the TAC was not the major force 
behind the schoolwide projects, technical assistance providers offered support for the district's efforts 
as the schools implemented their new schoolwide plans at the beginning of the Nine-Site Initiative, 
providing ideas for new instruction and organization, helping with evaluation, and taking some district 
staff to a national schoolwide project conference. 

Problems with Chapter 1 requirements lingered in some urban schools at the end of the third 
year of the initiative. In at least some schools, teachers and principals remained somewhat baffled by 
program improvement and unsure how to make the improvements that they sensed were expected of 
them. In one urban school, the principal acknowledged the TAC's help in understanding the 
requirements, but still claimed confusion: 

Testing is my biggest thing. I still don't know what NCEs are. At the beginning 1 
had no idea about what a program improvement plan was. 1 have still never seen any 
program improvement planning guidelines. At first, the only thing that made me feel 
better was that when we went to Washington [for the initial meeting about the Nine- 
Site Initiative], nobody there seemed to know what was going on either. 

This principal decided that the best strategy to get out of program improvement was simply to show 
the students how to score higher on the test; she requested a test-taking workshop for the whole 
faculty from the TAC. In Dade County, a primary focus of technical assistance during the first half 
of the Nine-Site Initiative was to help school and district staff develop their ability to underst^d and 
use student test data. In the second year, the TAC augmented this focus with a number of workshops 
on test-taking skills. 

In some schools, an apparent lack of understanding about Chapter 1 program improvement 
provisions was due not to confusion or misinformation, but to the integration of Chapter 1 program 
improvement with the district school improvement initiative. Several local school improvement 
initiatives explicitly linked improving Chapter 1 programs with broader building-level improvement 
efforts. In Baltimore, Detroit, and Dade County, schools were required to integrate their plans for 
Chapter 1 with overall school improvement plans, In Baltimore, we found considerable evidence of 
staff familiarity with the details of school improvement plans; principals explained that one of the key 
features of the planning process is that they were required to develop annual objectives and then 
delineate (I) the enabling activities that the school would engage In to achieve each objective, (2) the 
resources needed to conduct the activities, (3) a schedule of the activitiee, (4) the people responsible 
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for carrying out the activities, and (5) monitoring and evaluation procedures that would be used to 
measure progress toward each objective. Finally, schools had to indicate which of four possible 
funding sources, including Chapter 1, would provide support for the activity. 

In some cases, then, there is no real reason for schools to consider Chapter 1 program 
improvement as an effort separate from other school improvement efforts. What could be interpreted 
as inattention to the Chapter 1 program improvement mandate can also be the realization of the goal 
behind that mandate-to improve the education of disadvantaged students by ensuring that it is a 
schoolwide responsibility. An effort to combine planning for general school reform efforts with 
Chapter 1 program improvement has the potential to convey the message that the two can and should 
be entirely consistent. 

ChangM ln SgrvtM Drflvgrv 

Most of the schools we visited made some type of change in their Chapter 1 programs during 
the course of the three-year initiative (as did most of the comparison schools). Most teachers and 
principals, however, did not attribute these changes primarily to the technical assistance; they were 
more likely to ascribe them to the normal cycle of revision and to the Hawkins-Stafford Amendments, 
which sent a signal to schools that change in the service of improvement is good. In a few places, 
the TAC/RTACs helped inspire change, either by making concrete suggestions about possible changes 
or by imbuing school staff with enough confidence to create change themselves. In most cases, 
however, the technical assistance providers offered support for changes that probably would have 
happened anyway (and possibly helped schools achieve more than they might have been able to do 
without assistance). This section examines the types of changes in the delivery of Chapter 1 services 
that occurred in the study schools over the past three years. 

Overall, changes in service delivery were not accompanied by a measured examination of how 
current practice fails to accomplish the school's goals and how alternatives may better achieve them. 
The exploration of new service delivery models in many sites was approached on a trial-and-error 
basis. The change in service delivery model that we observed most often was a switch to a 
schoolwide project. While a few schools used this opportunity to try new instructional techniques, 
others used the additional resources to reduce class size or hire noninstructional coordinators, as the 
following examples indicate: 

• Both Detroit schools that we visited became schoolwide projects in the first year of 
the Nine-Site Initiative. One school used the opportunity to hire a full-time staff 



33 

4a 



coordinator, who most recently worked as a technical assistance provider in one of six 
district area offices, We did not discern any major changes to Chapter 1 service 
delivery in this school due to the schoolwide project; Chapter 1 teachers still pull out 
their students. In the other school, also a schoolwide project, the Chapter 1 teachers 
began to experiment with instruction in the regular classrooms instead of relying 
solely on pull-outs. This was not attributed directly to the technical assistance; 
however, the TAC staff member was involved in the school improvement planning 
activities and probably had some input into the decision to change. According to the 
Chapter 1 teacher, some teachers were resistant to Chapter 1 instruction in the 
classrooms and preferred their students to be pulled out. 

• In the BIA site, one school had been a schoolwide project since the beginning of the 
Nine-Site Initiative; all classrooms had a Chapter 1 aide, and students were net pulled 
out as frequently as they had been. In addition, for the past two years the school has 
used Chapter 1 funds to offer a five-week summer school to enrich the academic 
experience of students and provide staff development opportunities for teachers. 

• One participating school in Kentucky became a schoolwide project this year, using the 
extra funds for teacher training and visits to other schools. This school was unable to 
find enough of these added activities to spend the large allocation it received for staff 
development as a schoolwide project. In contrast, an energetic principal at the 
comparison school, which also became a schoolwide project this year, worked closely 
with his staff to identify a wide range of activities and quickly used the extra funds. 

Some of the schools we visited experienced problems obtaining or keeping schoolwide project 
status. One of the participating schools in Dade County (as well as the comparison school) failed to 
meet the three-year accountability requirements for schoolwide projects and reverted to a traditional, 
targeted program. The other school was anticipating the evaluation this year and was worried about 
suffering the same fate. The principals and teachers in schools that lost their schoolwide projects 
believed that the resulting increases in class size hurt the quality of their instructional programs. The 
TAC played a limited role in the implementation of schoolwide projects, although negotiations 
between the TAC and the district are continuing to identify TAC services that can contribute to the 
development of schoolwide projects during the 1993-94 school year. One of the BIA schools applied 
for schoolwide project status for the 1992-93 school year. However, the approval came too late for 
the principal to hire additional teachers for the purpose of lowering class size; instead, the school 
switched from a pull-out model to an in-class model, which has been unpopular among staff because 
teachers are unaccustomed to team teaching and uncertain what their roles should be in the classroom. 

One of the Mississippi schools was a schoolwide project until last year, but declined to 
continue the project due to a staff preference for pull-out services. RTAC monthly reports noted that 
several schools in the Nine-Site Initiative dropped their schoolwide projects because cuts in state 
funding made it impossible for districts to meet maintenance-of-effort requirements. The other school 
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in this site became a schoolwlde project at the beginning of the Nine-Site Initiative and was awaiting 
its evaluation results. 

In almost every school we visited that was a schoolwlde project, this facet of their Chapter 1 
programs was far more important to staff members than identification for program improvement. 
Meeting the third-year accountability requirements to ensure continuation of schoolwide project status 
garnered more concern and attention than did testing out of program improvement. In many of the 
schools in which the technical assistance addressed issues related to evaluation, these accountability 
requirements were the subject of much of the assistance. 

In addition to the switch to schoolwide projects, some schools began to move away from their 
narrow reliance on pull-out instruction, experimenting with in-class techniques and team teaching. 
For example, both schools we visited in Iowa reported such experimentation, which teachers said 
resulted from the close working relationships that developed between Chapter 1 and regular program 
teachers on the program improvement team. The Chapter 1 teachers in these schools still relied 
primarily on pull-outs; however, lessons from Chapter 1 teachers were much more likely to 
complement lessons in the regular classroom than they were prior to the RTAC services. Some of the 
changes in service delivery that we observed were unrelated to the Nine-Site Initiative; for instance, 
one school in Chicago expanded the use of computer-assisted instruction fjt Chapter 1 classes, with 
technical assistance provided primarily by the software vendor. 

Teachers 



Impact on Teachers 

In most of the schools we visited, the bulk of the time spent on technical assistance directly 
involved teachers and often included principals, Chapter 1 coordinators, and teacher assistants. The 
impact of the technical assistance activities on these participants varied widely from site to site and 
from school to school, for a number of reasons. 

First, as Section II of this report indicates, exactly who participated in a particular school 
often varied from one technical assistance activity to another. In schools in Iowa and Baltimore, a 
stable team of teachers participated in every activity over the course of the year. This pattern of 
consistent participation made the technical assistance a fairly big deal in the lives of the participating 
teachers. It also provided the teachers with a built-in support network, fortified by frequent 
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interaction and shared experiences, as they worked to try new instructional strategies. In contrast, 
teachers in Kentucky and Chicago reported that the technical assistance providers usually met with 
different teachers on each visit to the schools included in our study. Thus, regardless of how many 
hours the technical assistance providers spent in the schools, the activities had a low profile and low 
priority for most teachers, because they happened only occasionally at best. 

In addition, we did not observe a multiplier effect from the technical assistance. The impacts 
on teachers who participated in the activities fairly frequently and regularly did not extend to their 
colleague* who were not involved with TAC/RTAC activities. Teachers in some sites made it a 
priority to share their experience and information with the rest of the faculty but found it difficult to 
create enthusiasm or momentum second-hand. In a number of schools, principals mentioned plans to 
have teachers who participated in the technical assistance train other teachers in order to create a 
"ripple effect" and extend the impact of the technical assistance activities. However, we did not hear 
of any specific instances in which these plans were carried out. 

Second, the principals' support appeared to be an important, although not necessarily 
essential, factor in determining impact. For example, the principal at one BIA school was an active 
consumer of technical assistance, organizing the school to best receive and make use of assistance for 
ongoing reform efforts and coordinating assistance activities. She maintained that the RTAC's single- 
minded focus on Chapter 1 program improvement facilitated her vision for the school; it allowed her 
to attend to the myriad responsibilities that principals face while knowing that the RTAC would help 
her maintain a focus on improving instruction. Teachers in one Iowa school believed that the strong 
support of the principal-who attended all workshops with his staff, freed team members for all 
activities, and worked to increase inservice staff development time-gave them the confidence to 
experiment with new strategies and innovations they might not have tried otherwise. In two other 
sites, where the comparison schools were making more progress toward school improvement than the 
participating schools we visited, the principals of those comparison schools actively sought outside 
assistance and professional development opportunities, working in conjunction with teachers. They 
also provided opportunities for teachers to share the new ideas with colleagues through workshops and 
classroom demonstrations. 

We also saw several examples of a principal choosing to stand aside and let the technical 
assistance provider work with teachers. One Baltimore principal commented, "I could never have 
made this school faculty competent in cooperative learning. (The TAC] did that," implying that he 
had too many other responsibilities to devote the amount of time to teaching and coaching teachers 
that is required to make lasting change. In the other Iowa school we visited, the principal permitted 
the team teachers to participate fully in all technical assistance activities. However, the school had a 
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separate program funded through another federal grant to which the principal accorded greater 
priority. The principal reported thai she had made no plans to provide time for the team to continue 
to meet together or attend workshops as a group after the conclusion of the Nine-Site Initiative. 

In schools where either the principal or another person was considered the instructional 
leader, the TAC/RTACs sometimes had limited opportunity to work closely and frequently with 
teachers. One Detroit principal hired a staff coordinator with schoolwide project funds. The 
coordinator strongly believed that it was her role to provide whatever technical assistance teachers 
needed. In this school, the TAC staff member spent a substantial amount of his time working with 
the principal and the school improvement team on administrative and classroom management issues. 

Teachers' Criteria for Judging the Value of Technical Assistance 

Teachers had several criteria for determining the value of the technical assistance, all of which 
focused on short-term changes and quick judgments about usefulness. First, most teachers liked the 
new ideas they learned, finding them interesting and relevant to their classrooms. They told us that 
most of the ideas they picked up through the technical assistance were things they could use right 
away (e.g. t how to regroup using manipulatives to teach addition and subtraction, having students 
keep a "buddy journal" while reading a literature book, and the use of "webbing" as a p re-writing 
activity)* They particularly appreciated the accessibility and personalized nature of the on-site 
assistance, seeing it as a welcome contrast to district staff development programs that consist of a 
room full of hundreds of people listening to a lecture. In most of these schools most teachers were 
not hostile to the new ideas and information contained in technical assistance activities (although there 
were usually the stereotypical "resistors" who dug their feet in and shut their doors). 

One major contribution of the TAC/RTACs in many sites was sharing information and ideas 
from other states and districts. Many teachers told us how much they appreciated learning what goes 
on someplace else, partly because it helped them feel less isolated and partly because they felt their 
own, local ideas might have been stale. The infusion of new ideas helped fuel enthusiasm for 
program improvement in some of the schools, particularly in those rural schools where we observed 
such enthusiasm. For example, virtually all the teachers and administrators in the participating 
schools in Mississippi grew up in the area and attended the local state college; therefore, workshops 
given by Delta State faculty, which comprised most of the non-RTAC technical assistance, offered 
little in the way of fresh ideas. As a result, the link to programs outside the region supplied by the 
RTAC helped secure teachers* interest in RTAC services. In Iowa, the RTAC brought a team of 
teachers and administrators from a whole-language-bucd school in Ohio to serve as a model at the 
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beginning of the Nine-Site Initiative. This kind of exposure encouraged teams to request visits to 
other schools to get new ideas and see models of good programs; teachers from one Iowa school, for 
instance, traveled to Hannibal, Missouri* to visit a school implementing a whole-language, integrated 
language arts program, 

Second, in the sites where TAC/RTAC staff often worked in the schools with teachers 
individually (including Kentucky, Iowa, Detroit, and Baltimore), teachers appreciated the chance to 
see how new instructional strategies worked in their classrooms with their students; demonstration 
teaching was often mentioned as a favorite strategy. Teachers value demonstration lessons because 
they permit them to build craft knowledge through observation and discussion of in-class logistics 
with other teachers (as opposed to reading a research article or discussing theory). One teacher put it 
this way: "I can get the theory on my own if I want it. [ like staff development that gives me ideas 
of lots of things I can do tomorrow with my children." 

As testimony to the value of demonstration teaching, one Chapter 1 teacher said, "She knows 
my kids so she can offer [appropriate] material." Another commented, "The person who is assisting 
should come in and do actual teaching and model lessons. Any teacher can do what she sees done 
and incorporate it into her lessons." One teacher in an urban site said: 

I find this very helpful. You can talk all about a teaching strategy, but once the kids are 
there, anything can happen. It's good to see someone try it with your students. Then if it 
works well, of course you are going to do it. 

In addition to demonstration teaching, teachers liked receiving (free) new materials; one 
teacher in a rural site said, "I was tickled to death to get supplies that I could use right away in my 
classroom." Another teacher said, "I have stacks this high of strategies, things 1 can do with the 
kids. ... I have new materials-it's a new lease on life." 

Third, as discussed in Section II, teachers saw TAC/RTAC staff as "outside experts." In 
some sites, this gave them a lot of credibility. Teachers and administrators in participating schools 
seemed convinced that federally funded technical assistance can bring many more resources to bear 
than local efforts. They saw TAC/RTAC staff as better prepared and more expert than local trainers. 
One teacher in a rural site echoed the opinions of a number of teachers: 

[The local state-sponsored staff development agency] tries, but they can't afford quality. 
They don't do quality. We've seen plenty of bad workshops. . . . With the RTAC, it's 
crystal clear. These people have experience. 
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One rural principal said, "Sometimes a consultant, an outsider, has more credibility, If the person's 
traveled more than 100 miles, you have to listen to him." Yet, an RTAC director bemoaned what she 
saw as an unnecessary dependence on outsiders for assistance: "Principals are less likely to schedule 
an inservice just [for teachers] to talk. Somehow that's less impressive than someone walking in with 
a briefcase, " One principal noted that the TAC had the same effect on parents: "Parents tend to pay 
more attention to things that they hear from an outside expert. They listened to him even when he 
said things that we had already told them." Teachers and principals also appreciated the perspective 
of an outsider who could examine weaknesses in the school with a clearer eye than school staff. One 
principal said, "When we ran into a problem, she was a problem-solver, a catalyst. ... If there 
were situations that she, as an outsider, could deal with more safely, then she did. She made it easier 
for me to deal with some things." Another principal commented: 

TAC has been very supportive. [The TAC staff member] has been my cheerleader. 
Each time she came, she would show us more about what was happening. She helped 
us understand and articulate our progress. Sometimes we didn't know how much we 
had done, but she helped us see it. 

In other sites, the "outside expert" tag was a disadvantage, when TAC staff worked with 
teachers who were accustomed to a constant flow of experts who drop in, drop something off, and 
drop out of sight, The TAC working in Los Angeles wrote in their final report on the Nine-Site 
Initiative: 

Los Angeles Unified School District is very large, and is confronted daily with offers from 
many organizations, both public and private, to help heal what ails the system. On any given 
day of the week in the Nine-Site Project schools, there is an offer by some special project, 
some special initiative, or some special funding to combat the enormous problems the schools 
face. . . . These offers are viewed differently by each school. In the experience of the TAC, 
the first response is often one of suspicion. ... It was a tenuous task for the Region F TAC 
to convince schools that we were there to support their efforts and to build on what they were 
already doing in their schools-and not to bring in another outside project and layer it on top 
of all the other projects in the schools. 

A teacher in one urban comparison school said, "We do things best when we develop them. We can 
get more help from our neighbors down the street than from a consultant from far away. . . . The 
problems that we have here are our own. When you're trying to get it together, an outsider can make 
it worse." This teacher worked in a district where the teachers' union recently voted to call for a 
moratorium on all district and state reform efforts in an attempt to relieve teacher frustration and 
stress resulting from constant mandates for reform with no additional resources. 
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Fourth, teachers appreciated having the same technical assistance providers return time after 
time to follow up on the assistance. Knowing that they would see the person again iiupired-or 
intimidated»soiT\o teachers to try new strategies. One teacher said, "[The RTAC staff member] asked 
us to write down three things we would do, checked up, and told us when we needed to improve." 
Another teacher in this site added, "She came in when I was doing a traditional lesson, and I nearly 
died." A teacher in Detroit aaid; 

What I liked is the follow-through-she checked. It was like, here she is again. We see it's 
not going to go away so we might as weli use it. That's what happened. You see it working 
with someone else so you try it. 

Fifth, and perhaps most influential, teachers almost universally liked the technical assistance 
providers personally and rspected them professionally, which added to their inclination to enjoy the 
activities. In Iowa, Kentucky, Mississippi, Baltimore, one Detroit school, and one Chicago school, 
teachers and principals consistently mentioned the individual TAC/RTAC staff when explaining why 
they considered the technical assistance valuable. Comments included the following: 

• "It's the way it was presented that waa influential. They showed us good results, 
materials, and ideas. You have to have someone with [the RTAC director's] 
personality. ! thought, if [the RTAC director] is there and has enthusiasm, then this 
is good." 

• RTAC staff answered my questions and went the extra mile to help us." 

• "They're a good friend. They've been real nice. They go the extra mile to help. 
Anything we ask for, we get. We have a real good feeling about them." 

• "They were always supportive and encouraging, and they never talked down to us or 
told us what to do or that what we were doing was wrong/ 

• "She is sort of like a mother to us. She reminded me of my first grade teacher, my 
role model." 

• "[He is] a member of [our family]." (This comment came from the principal of a 
school that has not been a particularly enthusiastic participant in technical assistance 
activities.) 

• "Other consultants talk down to us, like we're stupid or backward. [The RTAC staff 
member] is tike one of us," 
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In many schools, the TAC/RTAC helped enhance the professional image of teachers, 
convincing administrators as well as teachers that teachers were capable of improving their work and 
interested in performing as well as possible. Teachers and administrators in the sites in which 
teachers had frequent contact with the TAC/RTAC staff noted that the increased confidence of 
teachers who participated in the technical assistance was a major factor in their willingness to try new 
instructional activities and practices* perhaps more so than the specific content of the technical 
assistance. One teacher said, "It's not so much what she did, but she showed us it was all right to 
try new things." Another commented, "Three years ago, I was waiting for something different. I 
was bored, but didn't know there were other options. This is more fun. I hated worksheets. 
Someone gave me the okay to do what I lovw to do/ In both Iowa and the BIA, we heard that the 
technical assistance helped teachers feel confident enough to "let go of the basals 1 * and move into 
more literature Sased teaching strategies. By contrast, in one school where not much progress was 
made, a teacher commented that "people here don't believe change will happen." 

In Kentucky, where the impetus for instructional change came from KERA, not from the 
RTAC, the primary impact of the technical assistance was support for teachers as they worked to 
understand their new responsibilities. In a few instances, the assistance helped teachers overcome 
initial reservations about using new approaches (for example, RTAC staff convinced teachers that 
noisy classrooms that result from students working together can be good). 

In the two schools we visited in Iowa, one notable outcome of the technical assistance process 
was that all teachers on the team expected more of themselves, holding themselves-and each other~to 
. higher standards. This was true for Chapter 1 teachers as well as others; just as Chapter 1 students 
were included in the effort to move away from basals into literature-based reading, Chapter 1 teachers 
were treated as equal members of the school improvement team. In one school, the Chapter 1 room 
was almost an afterthought to the school; the path to it included a detour through the gym, the music 
room, and several narrow, twisting hallways and steep stairways. Although this isolation symbolized 
the role of Chapter 1 in the school three years ago, the Chapter 1 teachers were full participants in 
the technical assistance and at the end of the initiative strove toward the same goals as their 
colleagues. 
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The degree to which teachers are encouraged to interact professionally, receive the support 
and the time they need to do so, and see value in the interaction is another indicator that a school has 
the capacity to sustain-and maybe even generate--Ghange. This was not always the case in the 
schools we visited. An early TAC monthly report to ED identified several features in the 
organization of an urban district that influenced choices about how assistance would proceed: 

These schools are separated by long distances and congested travel routes within the urban 
area so combining them for services is difficult if not impossible. In addition, the schools 
have varied schedules and lack time in those schedules for staff meetings and staff 
development. In some schools, teachers are with their students for the entire day including 
lunch. The only time available for planning and meeting is the last hour of the day, a time 
that is not particularly conducive to learning or activities requiring concentration. Most 
schools do have 30 minutes available in the morning for school activities, but a 30-minute 
block of time is not adequate to conduct meaningful assessment or development activities. 
Contractual considerations prohibit extending the school day for staff development; however, 
staff in some schools do volunteer to work on their own time* 

Schools in some sites had more flexibility in their schedules than the schools in this site; 
nevertheless, lack of time was a constant concern everywhere. Some schools were able to get around 
the problem by hiring substitutes or having teachers and principals cover classes so that teachers could 
participate in the assistance activities. In other schools, either it was not possible to adjust schedules 
or no effort was made to do so to facilitate participation. 

In a few of the schools we visited, we saw evidence of more teacher interaction and higher 
degrees of collegiality than staff reported existed prior to the Nine-Site Initiative. In some places, this 
was due largely to increased levels of basic coordination and communication between Chapter 1 and . 
regular program teachers about lesson plans and student progress. In others, the interaction was 
deeper; teachers met to discuss instructional issues, coordinate their plans for improvement, and learn 
new techniques. 

In Mississippi, the RTAC and school staff maintained that staff cohesiveness had changed 
dramatically over the past three years. Teachers reported that three years ago, staff relations were 
marked by tension and adversity, with teacher aides critical of teachers and vice versa. In contrast, 
teachers told us this year that they were proud of the fact that they shared materials and ideas in 
meetings and used each other as instructional resources. Teachers also reported making more use of 
aides in class. In addition, teachers were more likely to share what they learned in workshops with 
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other teachers. The comparison school appeared to ei\joy little of the collegiality that was developing 
in the participating schools. 

In Baltimore, the technical assistance revolved around the formation of study groups, which 
remained in place throughout the initiative. These teachers, who taught the same subject at the same 
grade level, were able to form a tight network that offered companionship and support as the teachers 
struggled with a new state math program. 

In the best example of improved collegiality, the formation of teams in Iowa laid the 
groundwork for change in the schools by giving teachers a forum to develop relationships, share 
information, exchange support and critiques, and discuss emerging concerns. During workshops in 
Ottumwa and the round-trip van rides to get there, team teachers discussed the content of the 
workshops, explored their plans for program improvement, and developed both professional and 
personal relationships. These teachers did not have forma! joint planning time other than the technical 
assistance activities; the time they spent together through the Nine-Site Initiative allowed them to 
discover each other as resources. A Chapter 1 teacher said, "Before, they didn't know what we were 
doing. Now, they see us as a support-type thing, net just another place to pull your kids. " Principals 
in both schools steered new teachers to the team teachers to get information about their practices and 
guidance on instruction. However, the lack of joint planning time in the schedule will make it hard 
for these teachers to continue to meet when their monthly trips to Ottumwa stop. Teachers hoped to 
continue some of the activities they started with the RTAC's help, including a monthly study group 
for Chapter 1 teachers from all participating schools. 

One school at another site offered an example of how a lack of collegiality and 
communication among staff can undermine school improvement efforts. One teacher noted that 
although the principal had gotten the schools into the Effective Schools Program three years ago, this 
was the first year that the staff were really aware of this and were actually involved in activities. 
Staff also reported tension, apathy, and alienation among the instructional staff and between teachers 
and administrators. In contrast, the principal at another school in this site made an active effort to 
encourage leadership among staff to build their capacity for sustained growth, creating school 
improvement committees to work on specific climate and relationship improvements along with 
academic activities. 

Like increased collegiality, improved school climate is a building block for school change that 
can facilitate substantive reform. Our observations indicated that several schools experienced 
improvements in morale and climate during the three years of the initiative. In some cases, teachers 
and principals told us that the infusion of new ideas and the repeated presence of outside experts 



43 

58 



generated enthusiasm among school staff. In the Mississippi schools, one of the RTAC's major 
accomplishments was to improve the physical plants of some schools, lifting spirits in the process. At 
one of the BIA schools, the principal reported that the new, positive attitude in the school was a result 
of the increased amount of staff development and training, much of which was provided by the 
RTAC. 

In a few schools, climate actually worsened, although we would attribute this to factors 
including school conditions, societal problems, and budget cuts rather than to the technical assistance. 
In many of the urban schools, where the flow of "experts" is constant and thick, the arrival of the 
TAC had little uplifting effect-in some sites, this cycle of descending and departing experts has bred 
cynicism about H another new program/ Teachers in Los Angeles schools were especially skeptical 
when the TAC lost its contract after the first year of the initiative, rendering useless the time- 
consuming process of laying the groundwork for assistance. In Dade County, the trauma caused by 
Hurricane Andrew distracted school staff and pushed technical assistance well down the list of 
priorities competing for attention. 

While problems chat affected children, teachers, and schools varied in severity from one site 
to the next, coping with them was very much a part of the professional responsibilities of all of the 
teachers with whom we spoke during the site visits. In some of the schools we visited, the frustration 
of dealing with these problems resulted in very low morale and an absence of staff cohesiveness. 
Teachers said they wanted help in these areas, which fall outside the realm of curriculum and 
instruction; one of their criteria for effective technical assistance was that it provide such help. 

Instruction and Assessment 

In most sites, much of the technical assistance focused on classroom instruction and 
assessment of student learning. In almost every school, we heard about at least a minimal change in 
these areas, as teachers learned of and tried new activities. However, our evidence leads us to 
believe that these changes did not often translate into efforts to rethink instructional philosophies or 
make lasting changes in instructional practice. 

Classroom Instruction 

In every school, most of the teachers who participated in technical assistance reported that 
they learned about new instructional techniques (and also, in some cases, new ways to organize 
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governance teams or involve parents, evaluate performance, and design Chapter 1 programs), and 
they believed the TAC/RTACs were very good at bringing them this information. Many of the 
teachers further reported that they actually tried some kind of new strategy. The most frequently 
cited examples of new activities or strategies included math manipulatives, literature-based reading 
strategies, and cooperative learning. 

Although most teachers were open to hearing about new ideas for specific techniques or 
activities, some did not find the information or new techniques useful and decided not to make any 
changes. For instance, in one school where the technical assistance activities focused on cooperative 
learning, one teacher told us that she believed that cooperative learning hurts students because they 
expect to be able to work together for tests. This teacher reported that the technical assistance 
provider had tried to convince her to change her mind, but although she occasionally let the students 
do projects together, she held firm in her refusal to change her practice. Others tried the new 
techniques presented and later decided to return to their original teaching methods. Again, on the 
subject of cooperative learning one teacher told us, "We've tried grouping-it's noisy. We're not 
doing it now. The kids tried to copy, or let one do all the work. At first it worked beautifully, but 
then they got to where they wanted to cheat a little, so we had to put a stop to that. " We saw no 
evidence that the RTAC tried again to convert this teacher. 

In some sites, teachers and principals were more willing to participate and try new things than 
they might otherwise have been because they perceived the quality of the technical assistance offered 
through the Nine-Site Initiative to be higher than that of other available assistance. In Mississippi, for 
example, a district administrator noted that because the RTAC offered better-designed staff 
development programs and higher-quality workshops and presentations than previous assisters, the 
Nine-Site Initiative helped sell teachers on the idea of professional development. The administrator 
said, "We finally convinced teachers that inservice is good. Now they're asking for inservices before 
and after school. 1 * The comparison school, by contrast, provided a striking example of underutilized 
staff development. 

In Iowa, teachers universally believed the RTAC workshops were an improvement over other 
sources of assistance; many of them told us that they were initially skeptical about the initiative, but 
after attending the opening workshop they looked forward to each activity, confident that it would be 
valuable. Chapter 1 staff in Dade County also said they learned about the value of school-level 
assistance as a result of their work with TAC staff. Referring to staff development provided by the 
district, one teacher in a comparison school said: 
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Most workshops are not us#ft)l, They ire hit or miss in terms of (heir content and general 
relevance. On-site activities would bi Interesting It has to be something that is worthwhile 
and of high quality. It has to keep your Mention. 

On the other hind, in several of the schools in Us Angeles, principals who were not convinced that 
the TAC assistance wouid be better than other available resources (namely, the Achievement Council) 
were unwilling to allocate professional development time to the Nine-Site Initiative activities. 

Many teachers noted that they received plenty of free materials through the TAC/RTACs- 
they hid more literature books in their classrooms, binders full of worksheets and articles about 
teaching, subscriptions to journals, or math manipulative kits. In Iowa, teachers noted that they also 
spent their personal and school discretionary fluids differently, choosing literature books more often 
than phonics materials and stocking their classrooms with research information and personal libraries. 

Beyond these impacts-teachers learned about specific new strategies and had some new 
materials~we saw little evidence that the technical assistance affected the teaching and learning in the 
schools we visited. The schools were replete with isolated examples of instructional experiments. 
However, we saw little evidence of changes in practice or beliefs. A very small number of teachers 
went beyond occasional experimentation with new materials or strategies to include new techniques as 
a regular part of their instruction or to redesign their lesson plans to fit into & new framework that 
included broad changes in practice. 

Cooperative learning, a common TAC/RTAC topic well-received by teachers, is a good 
example of an instructional technique that teachers can use either as a set of activities now and then or 
as the basis for deeper changes in practice or beliefs. In one school where cooperative learning was 
the focus of the technical assistance, we saw evidence of both kinds of impact. Most of the teachers 
added activities to their repertoires that involve ftudents working in groups, such s& creating groups 
for science lab experiments or holding math contorts between groups. A few teachers went further 
and had students work together In cooperative groups most of the time, with studems assigned specific 
roles like reporter and facilitator. These teachirra showed us that their desks were in groups instead of 
in rows, but with one exception they did not talk about why they made this change (beyond the fact 
that the technical assistance provider suggested it) and how their teaching changed as a result. Few, 
if any, teachers appeared to have actually changed their beliefs; most saw cooperative learning simply 
as a discrete new strategy, not as a fundamental change in the way students learn. 

Teachers in another school reported that cooperative learning as taught the TAC swept the 
school and that all teachers used it in their claMrooma, The TAC provider was m<ich more 

46 



61 



circumspect; he said that although many teachers liked the idea and enjoyed talking about it, "they 
aren't applying the concepta of cooperative learning/ Teacher interviews indicated that teachers 
learned specific teaching strategies that were taught to them directly. Only one teacher spoke of 
arriving at a new level of understanding about the learning process as a result of the TAC assistance. 
Study groups in one urban site were presumably intended to go beyond the level of specific activities 
and encourage teachers to devote serious and ongoing attention to the teaching of mathematics. 
However, when asked about the impact on their teaching, the teachers in the math study group at one 
school reported learning two specific strategies: how to regroup using manipulatives and "counting 
on." One teacher at the other school reported using writing during math now; another mentioned 
math manipulatives. 

Iowa stands in contrast to these two sites. During our interviews with participating teachers, 
none of them mentioned cooperative learning as something on which the RTAC had explicitly 
focused. However, visits to their classrooms, in most of which the desks were arranged in groups 
and students generally worked together with little teacher intervention, indicated that teachers 
instituted cooperative learning in order to facilitate the substantive changes they had made in their 
teaching. One teacher said, Three years ago, the desks were not in little groups. Now, there are 
kids on the carpet and in cooperative learning groups. It changed because of the [RTAC] program. 
It's okay-they don't have to sit quietly. 11 Another teacher commented that although the RTAC had 
not explicitly encouraged cooperative learning, "it fl"ws naturally from the literature units. We just 
started doing it. It makes sense." 

In fact, Iowa was the only site where we found evidence of teacher changes that transcended 
the episodic use of new instructional strategies or techniques. This was also the only site where the 
technical assistance, for all three years, was guided by an agenda (imported by the RTAC) that stood 
out as clear, coherent, iterative, and focused. The RTAC decided up front to do two things: create 
strong and stable teams in each school, and help teachers learn how to teach reading and writing 
differently. In these schools, the team teachers added many new language arts activities to their 
repertoires (e.g., story mapping, predicting); many reported almost complete changes in their 
instructional methods for reading and writing. Most other teachers in the schools used literature 
books at least twice a quarter (twice more than they used to) by the end of the Nine-Site Initiative. 
Team teachers had many more literature books in their rooms than they used to, and all of them had 
created binders full of RTAC materials about new activities and practices. 

Beyond this activity-oriented improvement, many teachers on the teams, especially in one 
school, significantly changed their practices across all or most of their instruction: they had their 
students working cooperatively in groups; they developed interdisciplinary units where students 
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worked on science, reading, math, and writing in the process of doing a project on, say, plants; they 
moved away from basals (completely or partially) into literature-based reading instruction. Finally, 
we believe that at least several of the participating teachers changed their beliefs about teaching. 
They saw their students as active learners and themselves as facilitators. They had more confidence 
in their own judgment and were more creative in planning instruction. They believed that teaching 
and learning reading in the context of literature-real stories-as opposed to phonics generates a 
valuable love of reading and writing. They thought more of their students' ability to deal with 
complex assignments (e.g., writing and publishing books). Both RTAC staff and the state education 
agency Chapter 1 coordinator, as well as the teachers, credited the consistent theme and integrated 
content of the technical assistance program (and the personal characteristics of the RTAC director) 
with its success* 



The TAC/RTACs worked on assessment issues in two ways. First, they worked with 
administrators, and sometimes Chapter 1 teachers, in some schools to help them use test data better to 
identify weak areas (a traditional TAC/RTAC specialty). In Dade County, for example, TAC staff 
spent a large proportion of the technical assistance hours working on testing and Chapter 1 evaluation 
issues with district Chapter 1 staff. In schools in Detroit and the BIA, TAC/RTAC staff helped 
prepare the third-year evaluation for school wide projects. 

Second, a few schools called on the TAC/RTACs as sources of expert assistance as they 
developed alternative assessments. In the BIA schools, Chapter 1 assessment was a sensitive topic, as 
many Native American educators felt their students were at a disadvantage when taking nationally 
normed tests. In response, one school was working to develop portfolio assessments; the RTAC staff 
assisted teachers in this effort, both in group training sessions and in individual consultations. 
Teachers responded well to the training, especially the individual assistance. The principal felt 
confident that portfolio assessment could directly address Chapter 1 desired outcomes and thereby 
contribute to program improvement efforts. 

In Mississippi, the RTAC worked with the testing coordinator in one local education agency 
to develop instructional management objectives that were coordinated with the Chapter 1 program. 
The RTAC then helped the district develop "mastery checklists" for teachers to use in the classroom 
to track students' progress; regular and Chapter 1 teachers met every nine weeks to review the forms 
and set goals. In Kentucky, one of the most visible parts of the efforts to implement KERA mandates 
was the development of student assessments; teachers worked hard to prepare students for the new 
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performance-based assessments. Along with assisting in this endeavor, the RTAC t working with the 
district, focused on developing a set of measurable desired outcomes that could be used to evaluate 
programs in preparation for the elimination of the CTBS test for Chapter 1 evaluation. The state, 
under a waiver from ED, used only desired outcomes to identify schools for program improvement in 
1993-94, during the transition to the KERA assessment system. Tests developed for KERA will be 
equated to national norms to fulfill Chapter 1 accountability in the future. This was the only case 
where we found evidence of TAC/RTACs working with districts to develop new assessments to fulfill 
Chapter 1 accountability requirements; however, assistance in several places did include looking at 
other assessment strategies as a technical assistance topic. 

Student Outcomes 

Some of the schools in the Nine-Site Initiative demonstrated gains in student achievement, 
while others remained in program improvement. There was wide variation in the extent to which the 
changes were attributed to technical assistance, to Chapter 1, to school improvement efforts, or to 
other factors. This section catalogues the experience of the schools with respect to identification for 
program improvement. 

Iowa was the only site in which both schools we visited tested out of program improvement 
after the first year. This did not affect their zeal for improvement, however; their momentum 
surpassed the initial impetus of identification for program improvement, Nationally normed test 
scores did not appear to be a cause for much concern in these schools; one principal was not entirely 
sure what his school's program improvement status was. A local test did carry some weight; some 
teachers in one school were unwilling to move away from basals completely because they felt that 
students needed basic decoding skills to do well on that test. 

In Baltimore, the BIA, Chicago, Los Angeles, and Mississippi, both study schools we visited 
were in program improvement during the third year of the initiative. In Kentucky , otltf school tested 
out at the end of the first year and was identified again st the end of Ihu second year of the initiative. 
A second school had tested out of program improvement by the third year of tile initiative. An 
administrator in one BIA school implied that the technical assistance might have actually contributed 
to the failure to improve achievement, at least in the short term. While noting lhal his stuff has had 
more professional development over the past four years than in the previous 1 7, he complained that 
the wealth of staff development may have had an adverse effect on student achievement because of the 
amount of time teachers spent out of the classroom. 
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In Dade County, three participating schools tested out of program improvement during the 
Initiative's second year. In its summary report to ED, the TAC claimed credit for "a SO percent 
success rate" among the six schools it served. In another report, the TAC concluded that this was, in 
part, the result of "creating and maintaining the necessary awareness ... for changes to take place," 
including identifying weak areas and suggesting alternative instructional approaches. District 
Chapter 1 staff had a different perspective; they saw the TAC contribution as "sensitizing teachers to 
the need to help their students develop test-taking skills. " 

In two sites, other evaluation requirements carried more weight than program improvement. 
Both Detroit schools were coming up on their third year as schoolwide projects and were preparing 
for that evaluation. One of the schools, which tested out of program improvement after the 
initiative's first year, was also a participant in the Detroit Compact, which evaluates schools 
according to grades, test scores, attendance, tardiness, and other measures. In Los Angeles, schools 
were under a state mandate that extends program improvement for three years in order to avoid the 
common phenomenon of schools bouncing in and out of program improvement planning 
requirements. Of the schools we visited, only the comparison school showed improvement in scores 
over the three years. 

Teachers in some of the sites were quick to point out that despite continued low test scores, 
they saw evidence of student improvement in other areas. At one BIA school, teachers reported 
improvements in their students' writing ability. In addition, attendance and homework completion 
rates increased in response to incentives (e.g., banana splits and parties). The principal in one 
Baltimore school noted that ,f (TAC work] has not yet manifested in student achievement, but I can see 
a change in the attitudes of students/ including increased metacognitive skills, a reduction in 
disciplinary referrals, and more efficient math computation skills. In Mississippi and Iowa, teachers 
imparted anecdotal evidence that children were reading more and had more positive attitudes toward 
reading; teachers in both sites attributed these changes to the technical assistance. 



District and State Administrators 

In several sites, significant portions of the technical assistance services were devoted to 
working with state and district administrators. Dade County was the most striking example, during 
the third year of the initiative, 70 percent of the hours spent on technical assistance were focused on 
the local education agency. In several other sites, TAC and RTAC staff spent smaller but still 
significant percentage* of their time with administrators. As was the case at the school level, we 
found that the technical aaaUtanct had more impact on raising awareness of Chapter 1 regulations and 
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program options than on actual changes in the organization of district Chapter 1 programs, the way 
that local administrators work with schools, or the use of technical assistance in the site. 

In two of the rural sites, the technical assistance did help engender changes in district/state 
administration of Chapter 1. In Kentucky, Chapter 1 testing looked different than it had in the past. 
As we noted above, the RTAC helped the district make the transition from norm-referenced testing to 
the use of desired outcomes for the purpose of identifying schools for program improvement. In 
Iowa, where the state Chapter 1 coordinator took an active role in the technical assistance, his 
experience with the Nine-Site Initiative led him to rethink the way the state Chapter 1 office provides 
technical assistance to schools throughout the state. During the third year of the initiative, he formed 
two other clusters of schools similar to the cluster that participated in the Nine-Site Initiative. The 
RTAC ran the kick-off team-planning workshop for each of these clusters, using resources available 
through its regular contract with the state. State Chapter 1 staff will conduct the activities similar to 
those conducted by RTAC in the participating schools. A third cluster will follow next year. Both 
the state coordinator and the RTAC director expected the state contract with the RTAC to be 
structured very differently than it has in the past as a result of the positive experience of the Nine-Site 
Initiative. 

In Mississippi, RTAC staff helped administrators in two districts that merged develop new 
Chapter 1 data and recordkeeping systems and begin to align Chapter 1 curriculum with that of the 
regular programs. RTAC staff also helped the new district develop a unified approach to delivering 
Chapter 1 services. 
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IV, CONCLUSIONS AND NEW DIRECTIONS FOR TECHNICAL 

ASSISTANCE 



Many federal and state technical assistance programs are organized to convey information 
about options for improving instruction or some other element of educational practice. Some 
technical assistance goes a step further to include a review of implementation strategies for using the 
information. In either case, the assistance providers are "experts" who transmit information and 
impart skills to their audiences. 

Most educators and sponsors of assistance expect this approach. For them, good assistance 
effectively and efficiently conveys useful information and provides some guidance about how to use it 
in daily practice. Good assistance also does not disrupt the schedule-or, if it does, the trade-off is 
worthwhile because of the quality and usefulness of the content. Ineffective assistance, on the other 
hand, is assistance whose content is not useful for the target audience or assistance that seriously 
disrupts other important work. 

The assistance provided under the Nine-Site Initiative was, within the narrow limits of this 
model, generally good assistance. The TACs and RTACs understood local demands for assistance 
and worked hard to meet them. One indicator of their success in meeting local demands is that in 
five of the nine sites, negotiations are under way or contracts have been signed to extend TAC/RTAC 
assistance into 1993*94. Our findings about the amount, content, organization, and outcomes of the 
assistance lead to several favorable conclusions: 

• The technical assistance activities and services generally met teachers* and principals' 
standards for quality and were therefore considered effective. 

• The TAC/RTACs provided the schools with a large amount of information and 
materials about instructional practices, curriculum, and student assessment. 

• Many of the teachers who participated in the assistance activities reported that they 
had experimented with the new strategies in their classrooms and found that these 
strategies "worked"--they held students* attention, they relieved teachers' boredom, 
and they offered new entries for lesson plans. 

• Follow-up visits, although usually limited in duration and frequency, gave teachers a 
chance to review progress and seek additional advice on how to use the information. 
Repeated visits over the three-year period, even though they were not always frequent 
or regular, conveyed the sense that the TAC/RTACs were committed to working with 
the schools and that they understood local needs and concerns. These long-term 
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relationships helped set TAC/RT AC assistance apart from assistance that teachers and 
principals reject as not helpful. 

• In several sites and schools, TAC/RT AC assistance extended the standard model in 
two additional ways. First, it was targeted to stable groups of teachers whose 
participation was sustained for a long period of time. Second, the assistance topics 
were cumulative-each individual activity was part of a framework that connected it to 
a broader vision. 

In short, the assistance provided under the Nine-Site Initiative conformed to a generally 
accepted and reasonably popular model of assistance as the transfer of skills from experts to 
practitioners. This model is directly analogous to staff-development approaches that train the 
participants in specific skills. Although these models of technical assistance and professional 
development currently prevail as the most popular, they have serious shortcomings. There is 
widespread agreement within the education policy and research communities that a transfer of 
knowledge and skills is rarely sufficient to help schools improve In meaningful, lasting ways. Little 
(1993) makes a distinction between transferring specific skills and fundamentally reforming schools: 

To fit opportunities for professional development to a campaign for the principled redesign of 
schooling is arguably a different matter from organizing the training and support to implement 
a program or a set of readily transferable practices, (p. 132) 

Little and others, notably Michael Fullan (1991, 1992), argue that fundamental school reform requires 
a transformation of the culture of schools and school systems. Teachers, principals, parents, and 
other members of the school community establish a new organizational culture conducive to initiating 
new practices and policies. Proponents of systemic reform go a step further to suggest that, if these 
local efforts are to have lasting significance, they must be encouraged and supported by leadership 
from the top. The leadership sets national standards and policy mandates to support their 
implementation. 

These more recent perspectives on school change differ from the earlier perspectives in at 
least three important ways. All of them have implications for planning and implementing technical 
assistance programs. At present, most technical assistance is organized around several assumptions: 

• The skills-transfer model defines the focus of technical assistance as isolated problems 
or issues in school practice, with solutions delivered by the technical assistance 
provider. 

t Because the solutions are brought in from the outside, school staff are seen as 
essentially passive receptors of the new ideas and practices. 
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• The impetus for charge also comes from outside, with the result that the external 
assister assumes a antral leadership role in the process. As a result, the process 
often collapses when the assistance ends. 

In Little's terms, this approach represents the "dominance of training over problem solving" 
(p. 143). The more recent cultural views of change stress the importance of school-based initiative 
and leadership, often supported and encouraged by policy mandates from the top. These perspectives 
on change include an expanded and redefined role for technical assistance, However, this perspective 
places the school itself at the center of initiating and directing the change process, while the technical 
assistance provider moves away from the center of decisionmaking and authority. 

In our view, critics of conventional staff development and technical assistance have identified 
a crucial flaw in these approaches, one that inevitably limits their effectiveness. The Nine-Site 
Initiative, despite some real contributions, had a limited impact on participating schools specifically 
because the assistance conformed to the standard models of external assistance and school change. It 
was designed to support training and the transfer of information and skills, not problem solving. If 
the intent of the initiative was to generate serious attempts at Chapter 1 (or broader) improvement, 
then the traditional model of technical assistance was not a good choice. 

Hindsight is easy. At the time that the Nine-Site Initiative unfolded, there were several 
reasons why the TACs and RTACs were unlikely to pursue an alternative model of assistance. First, 
the recent history of government-sponsored technical assistance led teachers and principals to expect a 
certain type of service, most likely one that revolves around a set menu of options for workshops and 
presentations related to specific instructional techniques and how to install them into daily lesson plans 
(Fullan, 1991). They had been trained to appreciate and use this kind of assistance, and they had no 
incentives to question it. In addition, the fact that the initial technical assistance services generally 
followed a pattern-needs assessment, planning, and workshops-that very much resembled other 
school improvement programs and external assistance led principals and teachers to think about them 
in much the same way that they thought about these other activities. 

Further, given the link between the assistance and Chapter 1 program improvement, schools 
were unlikely to think much at all about what kind of help they needed. The general failure or 
unwillingness of most schools and district! to view Chapter 1 program Improve ment as much more 
than a bureaucratic event initially led them to be skeptical, or worse, about the possible contributions 
of the Nine-Site Initiative, even in schools where serious schoolwide reform efforts were under way. 
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The expectations of the participating schools were most conducive to conventional technical 
assistance. Moreover, the structure of the Nine-Site Initiative, its fairly hasty start-up, and the 
confusion regarding ED's goals in establishing the initiative gave the TAC/RTACs little reason-or 
opportunity-to rethink their traditional approach to technical assistance. Originally established to 
help state and district Chapter 1 administrators meet federal evaluation requirements, the 
TAC/RTACs have spent years developing suitable methods for working with large groups of people 
to transmit information about specific, limited tasks, with support and encouragement from ED. In 
the case of the Nine-Site Initiative, ED's strong suggestion that the Nine-Site Initiative begin with the 
TAC/RTACs cleaning up Chapter 1 test data to be sure that schools had been properly designated for 
program improvement, together with a sometimes uphill battle to establish themselves as credible 
sources of assistance, led the TAC/RTACS to stick with familiar strategies and topics and shy away 
from consideration of alternatives. 

In some ways, by the TAC/RTACs* own admission in many cases, the Nine-Site Initiative has 
been as much a learning experience for. the participating TAC/RTACs as it has for the schools they 
have assisted. At the outset of the initiative, they had precious little time to plan a brand-new and 
much more complicated technical assistance process while trying to understand local context and 
create niches for themselves in complex and troubled schools; in addition, they were still responsible 
for carrying on their other work. Hence, they chose the rational and reasonable course of following 
the technical assistance model of the day-their clients (ED) implicitly demanded it and their target 
audiences (schools) expected and happily received it. To expect something different, given the 
circumstances, would have been somewhat wishful. 

If ED's goal in sponsoring the Nine-Site Initiative was to identify effective strategies for 
school-level technical assistance to improve Chapter 1 programs, much can be learned from the 
initiative. Assistance activities in Iowa, Baltimore, and Los Angeles illustrate how relatively small 
amounts of funds can be used to provide assistance over a long period of time that is considerably 
more intensive than most government-sponsored assistance. These sites offer examples of assistance 
that targets groups of teachers and is planned and implemented in a cumulative and iterative manner. 
Services in several sites, most notably Kentucky, show the value of linking technical assistance for 
Chapter 1 to other state and local reform initiatives geared toward schoolwide reform. Activities for 
clusters of schools in Mississippi and Iowa are examples of how economies of scale can be achieved 
while maintaining the personalized nature of assistance. Assistance in all of the sites demonstrates the 
value of at least some follow-up visits and the importance of establishing long-term relationships 
between external sssisters and the target audiences. 
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On the other hand, the very fact of linking the assistance to a categorical program's mandate 
placed serious constraints on its reach and ultimate effects. Early experience in all of the sites 
demonstrates the problems of initiating mandatory assistance that is linked to compliance with the 
requirements of categorical programs. Administrators and teacher in all of the schools had little 
understanding of Chapter 1 program improvement at the time the Nine-Site Initiative began. For 
most of them, it represented a set of bureaucratic requirements that did little to inspire serious 
thinking about opportunities and directions for change. As we learned, the goal was to "to get out of 
program improvement. * Some schools did test out during the second or third year of the initiative. 
A few schools that demonstrated gains d not sustain them and were identified for program 
improvement a second time. Other schools remained in program improvement status at the end of the 
third year. We did not find any clear evidence to suggest that the changes in program improvement 
status were linked to anything that the schools or the TAC/RTACs did in the way of program 
improvement. 

Another way of stating this conclusion is that the organization, content, and, ultimately, the 
impact of the technical assistance were limited by the lack of clear and well-articulated visions of how 
the schools might be different. Although the Chapter 1 requirement for program improvement plans 
was intended to stimulate such visions, in fact the school plans did not reflect careful thought about 
curriculum, instruction, assessment, or staff development, nor did the plans appear to have emerged 
from extensive participation and deliberation by teacfr rs and other members of the school staffs. In 
the absence of luch visions or any incentives to develop them, teachers searching for new techniques 
and tricks welcomed and valued technical assistance that offered information about innovations in 
instruction, curriculum, and student assessment, and in some cases facilitated experiments with them. 
Similarly, assistance that helped administrators understand Chapter 1 regulations and student test data 
were useful, But its effects were modest; there were few noticeable changes in professional 
collegiality or schools' capacities to change or to be self-reflective. 

If ED's goal in sponsoring school-level technical assistance is to contribute to the fundamental 
redesign of education for disadvantaged students, the lessons from the Nine-Site Initiative are not the 
lessons of a success story . They suggest several possible new directions for ED to consider in 
designing future technical assistance efforts. 
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L Through SflBUMtiflM nrociirtn-i*. ™a nWi rrtno p^lremeiite. give assistance provider! 

t^tiYiijo m b^wd W DYtYtni informitilnn tbaat rawwafri fa^g ptflfauiflnii 

Federal support for all the providers of technical assistance should reflect clear expectations 
about the purpoiei of the assistance. By the same token, because assistance that nurtures self- 
reflection and critical thinking is largely uncharted territory-few providers and probably fewer federal 
officials know how to do it— the government should avoid excessive prescription about the content and 
methods of assistance. ED could encourage or require potential contractors to propose alternative 
models of technical assistance and explain why the models are appropriate for intended target 
audiences. In addition, potential contractors could be encouraged or required to provide evidence that 
potential target audiences are disposed to use the alternative forms of assistance. Indeed an important 
task in providing new kinds of technical assistance will be to help recipients develop the capacity to 
use it. 

The importance of these expectations could be underscored with new and different reporting 
requirements. ED could reduce or eliminate the current emphasis on aggregated totals (e.g., total 
number of people served, workshops, or telephone requests) and replace it with a system that includes 
information about implementing the new models, the difficulties encountered, and the extent to which 
these problems were overcome. A revised reporting system also would require contractors to provide 
the quantitative information necessary to support observations and assertions about implementation 
and effectiveness. The month-by-month reports that we used to collect information on services in the 
third year of the Nine-Site Initiative are examples, albeit very simple ones, of reports that permit easy 
analysis of assistance patterns and diagnosis of problems or gaps. 

2. Create a technical assistance network that connects to and leverages state and local support 
for profeolonal development 

In observing that a different model of technical assistance could have done much more to 
foster school improvement, we are drawing on the recent literature on professional development and 
its potential contribution to reform. Technical assistance in support of school problem solving must 
go hand-in-hand with more ambitious and unconventional forms of professional development. 

In fact, even conventional forms of technical assistance demonstrate the necessity of a 
connection with local professional development. Even when assistance is available to the target 
audience at "no cost" for the services, there are costs associated with participation. These may 
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include paying substitutes to cover glasses, paying stipends for participation during non-school time, 
or simply ensuring that normal allotments of staff development time are allocated to the assistance 
activities. The size of these commitments grows rapidly as they include opportunities for teachers and 
others to continue working together between visits by external assisters. Without such commitments, 
the contribution of technical assistance is seriously impaired. 

3, Create an assistance network organized primarily around themes or topics. rfl thg r thm 
categorical prftfrffllWi include both process and fflflfflf ffFttlfllfefff 

The experience of the Nine-Site Initiative demonstrated the problems that external assisters 
face when their assistance is linked to a program that is not central to local improvement efforts. A 
more compelling foundation for an assistance network would be to foster ambitious reform in the 
name of a broader goal-such as bringing all students to high levels of academic performance. 

By offering assistance from specialists in both process and content, policymakers would 
recognize that both are needed. The initial process tasks in school change include much more 
extensive attention to identifying local needs and creating a vision that will guide efforts to address 
them. Process specialists would initially help schools build norms of collegiality and cooperation and 
define goals, problems, and improvement tasks. Content specialists would then be available, at the 
request of schools and districts, to help solve the problems and carry out the tasks. 

In making decisions about funding, it would be necessary to balance support for those schools 
most in need with support for those schools that are ready to change. Schools that are not ready for 
change will require more of the early capacity-building services and represent greater risk of a low 
return on the investment than schools in which the norms of collegtelity and cooperation already exist. 

4. Integrate n ew technologies into the tfriinical assistance network. 

Recent advances in computer hardware and software offer a range of possibilities for 
exercising this alternative. For example, it would be relatively easy to include large amounts of 
printed text in on-line systems that are in place in many school districts. Video packages containing 
workshop sessions and demonstrations are also increasingly available at relatively low cost. More 
sophisticated interactive technologies have also shown great potential in a small number of 
experimental programs. 
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Nevertheless, the experiences of the Nine-Site Initiative suggest caution in using these new 
technologies. First, none of the schools we visited have the facilities to take advantage of much more 
than a routine electronic dissemination system, and even those facilities are available in limited 
numbers. Second, even when the facilities are available, we saw little evidence that they are used. 
An exception is the dissemination of information about KERA mandates and their implementation 
over a statewide educational television network: all of the Kentucky Schools receive these broadcasts 
via satellite transmission. Third, given the extent to which teachers valued the interpersonal 
dimensions of their relationships with TAC/RTAC staff, it is unlikely that they would avail 
themselves of the electronic dissemination services in the first place. Overall, even though the 
promises of this alternative are great, there is aUu the very real risk that the schools that need the 
assistance most are least likely to benefit from this element of the network. 

& Spypprt yawritY^nilding for the prwito ?f tahntaq gwfctanre. 

A central theme in these alternatives is that future technical assistance programs will provide 
services very different from those now being provided. Facilitating changes in the culture of schools 
will, for example, require assisters to establish long-term relationships and pay sustained attention to 
local issues and concerns. More specialized help in studying, implementing, and assessing recent 
innovations in instruction, content, and assessment require sophisticated understanding of the 
underlying theories as well as detailed knowledge of practice. Both setrof responsibilities, along with 
others that will almost certainly emerge, also demand understanding of and the ability to apply 
principles of sound consulting and effective adult education. 

As we saw in the Nine-Site Initiative, a shift in focus to school-level assistance combined with 
the need to establish and maintain long-term relationships with schools and districts were learning 
experiences for the TAC/RTACs. The alternative assistance strategies discussed here represent even 
greater changes in perspective and practice. Therefore, we expect that there will be a need for 
training and staff development opportunities to prepare technical assistance providers to carry out 
their new responsibilities. The on-line information systems being developed and used by the 
TAC/RTACs are but one example of internal capacity building. Networking activities among the ten 
Eisenhower National Regional Consortia for Mathematics and Science Education are a second 
example. More focused in-house staff development activities, including training and staff seminars, 
are a third possibility. We do not propose massive new investments in the professional development 
of assistance providers, but an exclusive reliance on "learning by doing/ with no built-in 
opportunities for reflection, is a formula for mediocrity. 
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In conclusion, it is certainly fair to say that the Nina-Milt Initiative effeuted a significant 
number of teachers, principals, district administrators, and technical asilitanee providers by Involving 
them in some conventional ways of working on ttautt rallied to Improving education for 
disadvantaged students. However, our evidence indicates thai for moat of IHmc participants, tha 
impact was not very deep and the effects of tha Initiative Will probably not l*»t lon|. Thua, the 
challenge for future technical assistance programs Is not to figure out ways io make tha traditional 
approaches more effective. Instead the challinga If to create and support technical assistance 
programs that recognize and address the complexity of school change as a cultural process and the 
professional needs and capabilities of educators. 
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